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1. The purpose of this report was to continue development of approaches 
to determining reading Vequir-ements in military jobs, following on^ 
rel'ated research' tinder Work Unit REALISTIC The feasibility of a 
general .methodology for determining 'litt^rac:^ demancs of Ar'- y jobs was 
assessed by means other than expensive, ^arge-scale job-^knowledge' and 
job-sample performance testing. - _ 

2. Three appro'acHes ibr assessing Army re readir dem^ndf^ - r-tudied: 

(a) determining reading grade level of d:-: ficulty^of MOS pr virHc: ::aterial; 

(b) developing a formula for estltoating reading .-grade level rr ^lity 
from the AFQT,."and determiniug a correlation between estimated readings 
ability and job proficiency;' *(c) correlating personnel reading skills with 
Job Reading Task' Tests proficiency for three MOSs^ "It was- cbncluded that 
remedial literacy training* should be aimed at producing no' le^ss than 
grade 7.0 -reading ability, c.id that" job reading requirements for any MOS 
can^ be established o'n the basis of existing 'Army Personnel data (AFQT 

and MOS/ET). The FORCASt readability formula that was develope^l provides 
a valid ^estimate . of -the reading difficulty of Anny technical reading 
material,^ and tan be us'^d as a quality-control device for~ simplifying 
reading material to match reading ability levels. • 

3. Per'sojinel who will be' interested^ in this report include researchers ' 
and instructors in training methods,, functional literacy, readability 
analyses, and job prof4.ciency . ' ' ' ^ 
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FOREWORD 



Work Unit READNEED was undertaken by the Human Resources Research Organi- 
zation in order to provide information about methodofogies for determining literacy 
demands of Army jobs. This repc-rt presents information on hree different approaches 
that may be usefui in determining ^.ow well personnel must i able ^to read-in order to 
do a job. Related r searches discu- 'd jn reports from HumRR Work Unit REALISTIC, 

The- research was conducted HuinRRO Division No Presidio of Monterey, 
California, where Dr. Howard H. Mt Funn is Director. 

Military supi : 'ft was'provided y the U.S. Army Training CV iter Human Research 
'nit. Presidio of hlonterey. Col. Ull: r n Hermann, Chief. 

The research uas performed by Dr. John S: C aylor. Dr. T^hoi" G. Sticht, Mr. Lynn 
C. Fox, and SP5 J. Patrick Ford, v ::h assistance by Mr. Wluiam . Hurckhartt and Mrs. 
Nina McGiveran. Military assistants : rorr. the Piuman Reseajre; Unii - SP4 D. Knderby, 
JFC Steven Snyder, and SP4 Jame^ 'Masters. 

'Special acknowledgement s ciu- : Dr.'R.O aidkc j2T jf ( ~y ^nn/i.ed 
i^^aluation Center, and to M^ )r -a": L. Bidde .th- "S. Amiy H^uiste . F-rsonnel 
.Sii:ppo!t JentQr, whoi • aid mau j pas;>ih- the research reporcea in Chapter 3. 

HamRRO research for the Department of th^ Army is conducted under Contract 
DAHC 19-73-C-0004. Army ^Training Research is conducted . under Army Project 
2Q062107A745. ' 



Meredith P. Crawford • 
' President 
Human Resources Research Organization 



SUMMARY AND CONCLUSIONS 



MILITARY PROBLEM ^ 

Major mobilization pf manpower resources is generally accompeinied by an influx 
into the Armed Services of men with marginal literacy skills. In order to make best use., 
of these men', they must be classified into jobs for which their limited reading- skills will 
suffice, or they must be given remedial literacy training to raise their reading skill levels 
to meet the job requirements. In either case, knowledge of the reading demands of jobs is 
necessary for the most effective action. 



RESEARCH PROBLEM 

The purpose of Work Unit READNEED was to study methodolorrief fc i rermining 
the reading requirr^-^^^^ntr f jobs. This research follows HumRHO Wor.. wi-. lijALISTIC 
(1), m Vi'hic. :eaoii_ ^quxrements . four major military occupational specialties were 
determined; in READNEED, certain of the approaches used in REALISTIC werb to be 
•refined, and the feasibility of a general methodology for determining literacy demands of 
jobs using current Army personnel files was to be tested without the expense of the ' 
large-scale job-knowledge and job-sample performance testing used 'in REALISTIC. 

■ ■ ' 

METHOD , / ' ' - ■ 

Threp approachef :or assessing MO'S. reading demands were studied: 

(1) A determination of the reading grade level -of difficulty 'of MOS printed 
materials using a readability formula especially designed and calibrated for use with Army 
personnel and Army job materi^s. 

(2) The development and use of a formula for estimating rfeading grade level of 
ability from the Armed Forces Qualification Test (AFQT), and a determination of the ' 
correlation between estimated reading ability and job proficiency indexed by the Primary 
Military Occupational Specialty /Evaluation Test (MOS/ET) and Enlisted Efficiency^ Rating 
(E^'R). All these scoresjcan be obtained from present Army data banks. 

(3) The- correlations of personnel reading 'skills vnth proficiency on Job Read- 
ing Task Tests (JRTT) developed in REALISTIC for the Cook (MOS 94B), Unit and . 
Organizational Supply. Clerk (MOS 76Y), and 'Wheel Vehicle Repairman (MOS 6i?C) 
MOSs. The e-ffects of selection and of Selection plus Combat Support Traininjg (CST)jon 
this correlation were evalua.ted by comparing the correlations of men tested at 'ijihe 
Reception Station at Fort drcP, with those of r^eri selected for CST in- the appropriate 

•MOS but not yet trailed (Pre CST group), and ot men frofft • selected and trained 
(Post-CSt)' in the appropriate MOS. ^ 

RESULTS ^ , ^ ^ V ^ 

The research on readability provided the following results: / ' 

(1) The production of a' simple, easy-to-admirlister formula for validly esti- 
mating the reading grade level of difficulty of Army job reading materials, based solely 
upon a'^coiint of the number of one-syllable words in a 150-word,/sample of the" material, *. 



This formula was named the FORCAST {FORd, C^lylor, STicht) formula, fc/Jowing the 
usual practice in readability Vesearch. - 

Information about the correlations among the Flesch, Duie-Chall, and 
.FORCAST readability formulas indicated that, although estimates of reading difficuity of 
materials based on these formulas are highly Correlated aiid of similar validity, the 
simplified FORCAST formula produced a more accurate estimate of the reading difficulty 
^of Army MOS passages. - - - . ^ 

(2) Information which indicated that more than half the reading materials in 
• each of seven MOSs exceeded the grade 11;0 level of difficulty as estimated b^^use of the 
FORCASt formula. 

The research . on the feasibility of using information in present Army data files to 
estimate the reading demand's of Army MOSs produced these findings: 

^ (1) Riading grade level (RGL)..and AFQT are highly correlated (rs ranging 
frpm .68 to .79). Hence the RGL may be estimatecf from AFQT scores with moderately 
high accuracy. 

(2) AFQT and job proficiency indexed by the Enlisted Efficiency Report 
(EER), ix lating made' by supervisors, were not^ related. Therefore, the EER cannot be 
used to estimate reading demands of jobs. This finding confirms previous observations on 
the relationships between supervisors' ratings and AFQT, reading ability, job knowledge, 
and job sample performance. Thus the EER shows no useful relationship to literacy or to 
the measurable components of job proficiency mediated by reading. 

(3) AFQT and the Primary Military Occupational Specialty /Enlisted Evaluation 
Test. (MOS/ET) were directly, and significantly related,^ indicating the feasibility of 
estimating the RGL requirements of MOSs on the*" basis of relationships among pegSmgN 
test scores, AFQT, and MOS/ET. / 

(4) RGL scores iissociated with proponent-established prof iciency cut-mf. scores 
for the MOS/ET for seven MOSs indicated that minimal passing scores 3etDyH;ffe^ 
proponent agencies in two MOSs were too. lo;w to estimate reading requirenients using 
this method; for five MOSs, the reading requirements so estimated ranged 'from grade 
levels 7 through 9. '1 

(5) Increasing the median, minimalj passing MOS/ET cut-off scores eight 
points— from 57 . (46%) points correct to 65. (52%) points correct— for seven MOSs, 
resulted in a two-year increase in median estimated RGL, from grade 9.8 to grade 11.9. 
This indicated that these MOS . tes^ts measure vnth great sensitivity in this t^iidrange of 
scores. * 

Research on Job Reading Task Tests (JRTT) indicated that: / 

(1) The. JRTT and standardized reading tests results were highly correlated, 
suggesting that these tests may be viewed as being equally valid, alternative measures of 
general reading Ability. Test-retest reliabilities for the JRTT were in the- rang j from .74 
to .8 5r-acceptable. reliability (stability) of scores on these experirhental instruments. Inter- 
correlations among the subtests of each JRTT with tfie sum of the remaining subtests in 
each JRTT were generally moderately high, and positive, iniplying that each subtest 
cannot be considered a measure of a sepairat^, independent reading skill. 

(2) Combat Support Training in the MOS for which a JRTT was designed 
improved perfbmiarice oh the JRTT for men of all reading levels from the 5th to the 
14th grade, although the greatestMmproyement was observed in the performance of men 
with poorer reading ability. ' 



' (3) In the Repairman's MOS (63C) and Supply' Clerk's MOS (76Y), selection 
for training in the MOS on the basis of classification test scores was associated with 
improved performance onr the JRTT for these MOSs; this was not true for the Cook's 
MOS (94B). , ' 

(4)- Correlations of JRTT, standardized reading tests, and AFQT with Post-CST 
academic grades ranged from a low of .49 to a high of .72. All instruments were equally 
effective as predictors of achievement in the MOS/CST program. 

. (5) For the Cook and Repairman MOSs, JRTT and AFQT scores were'signifi- 
cantly correlated with Post-CST performance on a job-knowledge, paper-and-pencil test 
designed in HumRRO Work Unit UTILITY to assess a job incumbent's knowledge of 
essential job information. Similar correlations obtained for Pre-CST men in the Cook 
MOS indicate that performance on the job-knowledge tests reflects, to a large degree, a 
man's reading ability rather than his job knov/ledge. The man who' reads better pre- 
sumably learns more and, therefore, scores higher on the job-knowledge test. 

(6) The. use of a decision rule stating that 80% of the Post-CST men at a 
reading level should get 70% dorcect on the JRTT for their MOS suggests that the 
minimum literacy requirement fbr Repairmen and Cooks would be grade 7.0, but lOth 
grade ability would be needed foj: Supply Clerks. 

CONCLUSIONS 

(1) Although iK^ single level of functional literacy can adequately represent the 
reading requirements 6p^he ran&e of MOSs studied, there appears to be a low'er limit of 
grade 7.0 for functional literacy in the Army. Almost no job reading passages wer6 found 
at or below this level in the reading materials prescribed for a. representative; range of 
seven high-density, but diverse MOSs' nor with minor exceptions, was the reading require- 
ment for any 'of 10 MOSs* established below ^this level by any of the procedures studied. 
Hence, remedial literacy training should be aimed at producing no less than seventh grade 
reading ability, and, optimally, should^ be, targeted to the level of a man's MOS 
assignment.^ 

-(2) Job reading requirements for ^y MOS can be established efficieritly and 

routinely on the basis of existing Army personnel data (AFQT and MQS/ET) with 
nominal effort and (:ost. Although reading ability is not usually assessed, the stable- and 
\substantial relationship (r^ .7) between AFQT and reading permits the estimation of the 
Mter without additional testing. ' 

^. (3) Job reading materials are, for the most part, so difficult to use that they 
are i^ijored by most of the intended users., .The,,easy-to-use FOHCA^Jjreadability formula 
provides a valid estimate of the reading difficulty of Army technical reading material for 
the Army population, and can be used as a quaIity-coritr61 device for the simplification 
of reading material ^o match the reading ability levels of the usdrs. 



^Three additional MOSs were added in order to assess the relationships between readings ability i 
established by standardized, grade -school referenced tests and performance on Job Reading Task Tests. 

^Current HumRRO Vfork Unit FLIT has as its objective the development of an experimental Army 
literacy training program designed to achieve seventh ^ade reading level. 
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Chapfer 1 
INTRODUCTION 



Rapid technological advances place a premium upon fundamental information 
processmg skills, especially language and literacy skills. If a technologically based organi- 
zation, such as the modern Army, is tp siurvive and function effectively and efficiently it 
must have personnel whose literacy skills match the literacy requirements of the jobs- 
within the organization. , 

^Although the Army has continually encountered the problems produced "by the 
scarcity of academically qualified manpower; these problems recently have been increased 
. _by:the mtroduction, under Pr oject 100,000, of large numbers of men whose readings skills 
fall substantially below the functional literacy levels of major i miUtary occupational 
' specialties ,(MOS) (1). Furthermore, it is anticipated 'that^as^the Army mofel toward an 
all-volunteer, force, a significant proportion of the, manpower will have to be drawn from 
the pool of less academically quaTifded worlcers. 
.^J Facing these problems directly,' ther Army has sponsored research by the Human 
^ Resources Research Organization to develop methodologies- for determining the literacy 
requirements of Army MOSs (HumRRO Work Units REALISTIC and READ NEED) and 
to develop a prototype literacy training program geared to produce MOS-reiated func- 
tional-literacy skills (HumRRO Work Unit FLIT). 

«The present report describes research performed in Work Unit READNEED, which 
was directed toward (a) the refinement of methods used in the REALISTIC research to 
establish literacy requirements of jobs, and (b) to develop more general, less expensive 
•■ , , methods. 1 . - ... ' ■ 

The REALISTIC effort used three different methods to identify reading demands of 
a small number of jobs. One method involved the identification of reading materials in 
five' Army MOSs, and an evaluation of the average reading difficulty level 'of thpse 
I materials using the modified Flesch readability formula of Farr, Jenkins, and Patterson 
* (2)1 This method, its limitations, , and its refinement under READNEED are discussed in 
V '^Chapter 2 of this-report. . " . ; . 

The second approach used in REALISTIC to evaluate reading requirements of Army 
MOSs^involved the measurement of literacy skills of some 400 men in each of four MOSs 
having a high-density input of low-aptitude m°en. These men were also tested on job- 
knowledge tests and four- to five-hour job-sample tests, in which cooks cooked vehicle 
mechanics repaired vehicles, supply clerks filled, out forms, and' armor crewtten operated 
^the tank and its ^yeapons systems. Relationships between literacy test performance and 
job kno^yledge and job performance were then studied to establish ,literAcy-skill levels 
/ associated with the various levels of proficiency on the job . tests. .While this approach 
yields a great deal of information, it is, extremely time consuming 'and expensive, and is 
not feasible as a general approach to establishing reading requirements for the hundreds 
of Army military occupational specialties in existence. Chapter 3 of this report {iesc'ribes 
an - alternative approach to identifjring reading-skiU. levels associated wilJi successful job 
proficiency^ Using data currently maintained in Army personnel files. • - 

In the final approach used in REALISTIC for establishing reading demands of MOSs, 
a detailed determination was made, by means of on-site interviews; of the job-specific 
reading materials actually used by job incumbents in three MOSs. For eanK nf thPRP 



MOSs, a Job Reading Task Test was constructed from these source materials, ^yielding 
tests consisting of reading material used by men in performing their jobs. As a final 
research activity under REALISTIC, Job Reading Task Test performance was studied in 
relationship to conventional school reading mea^sures and the AFQT. As a product of this 
effort, the Army was provided with (a) reading tests composed of the actual job reading 
materials in three MOSs, and (b) information on the relationships between performance 
on Job Reading Task Tests, general educational reading-grade level, and AFQT for men 
new tho Ar 'lo were tebted at the receptiru station at Fort Ord. 

provides ^ means of identifying the general reading levels associated 
Wi ous cx. oixon levels of proficiency on the-Job Reading Task Tests; ^'that is, the 

reading demands of a job can be stated in terms of how well a man needs to read {in 
grade school scores) in order to' read , job printed materials with varying degrees of 
-proficiency. Like the second, approach mentioned earlier, the determination of job 
reading requirements by testii^g men on job reading task tests is an expensive under- 
taking, and realistically could not be undertalcen for each Army job. Nonetheless, such a 
procedure is the most direct method for determining reading demands of a jol>, and 
might feasibly be. employed in conjunction with job clusters, using reading task tests 
constructed to represent the most f);fiqaently occurring types of reading in the cluster. 
Because of the potential of the .Job Reading Task Test methodology, psychometric data 
in addition to that collected in the REALISTIC research were obtained in READNEED. 
These data, concerning the reliability of Job Reading Task measures, and the influence of 
selection for and training in the MOS represented by the Job Reading Task Test, are 
presented in Chapter 4 of this report. ^ 

Finally, Chapter 5 presents , a general ; summary and a statement of conclusions 
regarding the problem of determining the literacy requii<^ments 15f jobs. 



Chqpfer 2 

DEVELOPMENT OF THE FORCAST READABILITY FORMULA FOR 
^RMY TECHNICAL MATERIALS 

ESTIMATING BEADABILITY 
. READABILITY FORMULAS " - 

P^or many, years, various formulas' for estimating the readability, of printed 
materials~~that is, how easy v they are to read ar?d understand—have been available (3). 
Generally speaking, indices of readability are established by following three basic steps. 

(1) A number of structural - factors, such as ^ the' average sentence length, 
number of syllables^ per word,'anct number of wor^s occurring with low 

■ ^ frequencies in general English. us^e, are identified. ' 

(2) The number of occurrences, of such factors in selected reading passages is 
Correlated ynth performance pn comprehension tests b^sed on the passages. 

(3) Formulas (multiple regression equations) are derived that state the ftinc- 
tional relationships between the structural factors and performance on the 
comprehension tests. ' 

For the average reader, a low readability score predicts a low level of comprehension 
^bf .the passage, while a high score predicts a high level , of comprehension. Sometimes this 
procedure is extended so that a formula will estimate the school grade level of students 
who reach a specified criterion level in answering comprehension questions after having 
read the p issage. It is possible, in this case, to state th^ reading grade level of difficulty 
of a publicuuon. 

There have been seversJ applications of readability analyses to military publications, 
with the objective of stating the reading levels needed to understand the materials (3,^4, 
and 5). In Work Unit REALISTIC, a /.modification of a formual devised by Flesch in 
1948 (1, 4) was used to assess reading difficulty of Army publications. The formula follows: 

Readability == (1.699 x the number of one-syllable words per 100 words)— 
(LOM X the average number of words Jn the sentences)— 31,617, 

This modified \formula correlates better than .90 with Flesch 's 1948' formula. The 
latter, in turn, has a "validity coefficient of .7.0 for predicting the reading grade placement 
at which 75% comprehension of 100-word samples of the McGall-Crabbs. Standard Test 
Lessons will occur (3, pp. 56-59). The raw-score" index numbers derived with the use of 
the above formula have a range from 0 to 100 for almost 'all" samples taken from ordinary 
prose. A score of 100 corresponds to the prediction that a child who has completed 
fourth grade' will be able to answer correctly three-^uai^ters of the test questions about 
the passage that is being rated*. In other words, .a score of 100 indicates reading matter^ 
. that is understandable for persons who have completed fourth grade and are, in the 
language of the U.S. Census, barely "functionally literate** (6, p. 225). Iri the REALISTIC 
research, the raw-score indices obtained with the Flesch J,formula were converted directly 
into school grade equivalents by means of a specially prepared table (5).^ 

•" ' ' ' ^ ■ 

^ *The Pale-Chall ReadabUity formula was applied to a subset of these materials, and the results are 
described in 5. 



The major usefulness of an appropriate readability index is that it permits an 
immediate estimation of the jeading ability level required to understand a passage,' 
making* use of clerical operations without the need for further testing. A readability index 
may be applied (a) to specific draft material in preparation in order to . gauge . its 
comprehonsibility for its intended audience, and (b) to samples of the job reading 
material in an MOS (as in the REALISTIC research) in order to determine the reading 
ability required to understand the job materials. Given an appropriate formula, all that is 
needed is the reading material, the readability definitions and formula, and a clerk 



coi:iipetent to apply it. 



LIMITATIONS TO USING AVAILABLE FORMULAS WITH zkHMY MATERIALS 

^v-Sever^-prolDlein5_ariseJn__a^ existing readability formulas to Army job reading 



mat^ial. General readabilifynformi^^ 
population; it is not known how appropriate these .indices may be for the young a'aiHF 
male Army reading population and for ^Army job reading material, with its characteristic 
style, format, and heavy use of ^echr\ical nomenclature. However, the fact that th^ 
formulas have validity coefficients of about ,70 for predicting the performance of school 
children on reading comprehension tests indicates that they accpuhC for roughly 50% of 
the variability- in reading performance o| children,- It is likely that they may account for 
less variability in adult performance, especially since material containing large numbers of 
technical terms would increase the estimate of difficulty / made by the readability 
formulas* " . ' 

An additional drawback' to /the use of general readability formulas with Army 
technical material is that some indices require special gramm.^tical or linguistic compe- 
tence on the part of the user, or the use of special word lists or' equipment that is not 
likely to be routinely and readily available to the general user. 

* ' . . 

PROCEDURES USED IN DEVELOPING THE. 
FORCAST READABILITY FORMULA 

. Although available readability formulas . have serious limitatipns for application to 
A^rny technical' m^^ the general approach to determining the reading skill level 

requirements of job printed niaterials by use of a readability index provides a relatively 
fast, inexpensive, and objective estimate of these requirements. Accordingly, the 
READNEED staff ^ undertook development of a readability index that, would be (a) based 
on essential Army job reading material, (b) normed for the young adiilt male Army 
recruit population, and (c) simple and readily applicable by standard clerical personnel 
without special training or equipment, The^formiila that was developed to these specifica- 
tions has been^designated the FORCAST Readability formula. „ 

The major steps in developing the FORCAST readability index for job reading 
materials included: . ^' 

(1) Determination of jobs (MOSs) to be included. 

(2) Determination of essential job reading materials. 

(3) Selection of reading passages from the job reading materials, and assessment 
of appropriate structural properties.' 

(4) Measurement of an individual reader's comprehension of passages from the 
. job reading materials. 

(5) Scaling of passages in terms of the reading grade level (RGL) required for a 
designated criterion level of comprehension. 



(6) Determination of optimal weights of the structural properties of passages in 
orde^r to maximize th^ prediction 'of RGL required to comprehend Ihe 
passage at the designated criterion level. ■ } 

Within this general framework, maJiy^ specific decisions made in tarrying out the 
general procedures had an effect on the Ijiitcome; these decisions are discussed in, detail. 



SELECTION OF MOS s AND ESSENTIAL JOB READING MATERt ^ " 

In developing the FORCASf readability index, essential job reading materials were 
collected for'seven MOSs: , ' 

, 11B20 Light Wea^Dons Infantryman ' \ 

26D20 Ground Control Radar Repairman ^\ 
6 3B20 ^ Wheel Vehicle Mechanic ' ' \ 

71H20 Personneil Specialist • - ^ \ 

76Y20 Armorer/Unit Supply Specialist ' ' " ^ 

~9IB20-MedicaL:Sp.ecialisl ' ./ * 

95B20 Military Policem^ ^ ^ '~ r — — - — . ' 

These MOSs were selected to provide information on high-density jobs over a Wide range 
of job families arid-^cpntent areas.. . = ^ 

The determination of what body of reading material is essential and integral to a job 
is frequently an arbitr^y one. Iii READNEED, the .definition reading i^aterial essential 
to job performance was less so because the DA Pamphlet 12- Series prescribes the source 
.reading material on which the 'mandatory* annual Primary MOS Enlisted ^valuation Test is 
based. For each MQS, the test study guide provides authoritative specification,, to the 
chapter level, of the source materials which a job incumbent must know to validate or 
maintain his job classification,, These materials -may^ be considered as essential job* reading 
materials, because they form the basic depository of knowledge that a. man needs to be 
designated as proficient and qualified for his job. / 
^ In order to select passages from which to develop the readability formiila, copies of 
he regulations and manuals identified by, DA Pamphlet 12- Series were obtained for the 
seven MOSs listed. The assembled job reading materials were sampled by scanning the^ 
pages for appropriate passages. Passages were deemed appropriate if they containeji 150 
words of prose (excluding tables) on either one ^ubject or' two closely related subjects. 
Passages were also^selected to repr^ent'the MOSs as eVefely as possible. ' ' 

Using the modified Flesch formula (2), -each symple passage, Was assigned a screening 
readability index value, . Twelve of these passages Were then sheeted to provide a full- 
range of readability, from the easiest to the; most di|ficult, encountered in the sampled 
. jo'b reading material. The§e passages are presented in Appendix A i 

" ' - ■ ' ■ ■ - .. , 1 ■ 

SELECTION OF STRUCTURAL PROPERTIES OF JEXT 

A literature search on tl>e -subject of readability yielded a Jiist of some 40- structural 
properties of passages that had been used in one readability^sformula or another. By . 
eliminating those "variables that were essentially duplicates or were not feasible for our 
purposes because they required specis^ competence or equipment, the candidate list of 
structural prop^rti^s of passages was reduced to 15, These variables, and the manner in 
which they were obtained are described below. ' ' \ i 

(1) Sentences : The number of sentences was counted, up tp'and inclufing the^ 
IBOfch word. of . each standard 150-word passage. The sentence containing the 150th word 



Was counted only if that word occurred in the latter half of that sentence. Each heading 
was counted as a sentence. ' , 

(2) Words per sentence : The number of word? in the sample G150) was 
divided by the number of sentences. 

(3) Independent clauses : The ^nmb'^" .1 clauses in 150 wordo 
was counted; An independent clause coiitaiiunt^ the 150th wurd was counted only if that 
word occurred in the latter half of that clause. :> 

(4) Words per independent cliause: The number of words (150) was divided 
by the number of independent clauses. 

(5) One-syllable words : The number of one-syllable Words in .150 words was 
counted. Syllabification was determined by the spoken language; for example, the 
number 14 v;as treated as the two-sylllable word ''*four-teen." Hyphenated words were . 
treated as a single word, an'd mere considered polysyllabic. Tn case of doubt, a dictionary 
was consulted.' _ j ' . 

— ^6.) Difficmic words : This wak the" number of words out of 150 that did not 
appear on the Dale list; of 3,000 familiar 1 words (5). Each difficult word was counted each 
time it appeared. - \ '. . i- '\ ■•' ■ - 

(7) Different difficult words :| The number of first occurrences of t\\e difficult 
words in 15G words was countedl 

(8) Df^^ "Words :-~The number-^f first occurrences only of words in 150 
words was recorded ' 

(9f) Three-or-more^syilable words : Thu was the nurii,ber of words of three or 
more syllables in 150 words. . 

(10) Totai number of syllables: The number of syllables in' 150 words was 
alculated. 

(11) Tot^ lett^irs : ,This was the namber of letters and digits in 15P words. 

(12) Syllables per sensence : The number of syllables in 150^ words was divided 
by the number of sentences. . 

(13) Letters per sentence : The p.umber of letters 'in 150 words was divicled by 
the number of sentences. 

' o (14) Seven-or^more-letter words : The mimber of words in 150 having more 
than six letters .or digits was counted? 

(15) Different three-or-more-syllable , words : The number ,of different words 
having three or more syllables in 150 words was'talKed. 

Each of. the 12 experimental passages (Appendix A) was) assessed' to determine its 
value for each of the 15 structiirilpropierties listed. These figures are shown in Appenxlix B. 

THE CLOZE PROCEDURE AS A MEASURE'OF COMPREKEIMSION ' . 

In the development of the FORCAST readability forniula, /an individual's compre- 
hension of the experimental ipassages was assessed by ^means^of the cloi:e test procedure 
(T). In constructing a cloze test, every fifth (or ?ith) word of /a passage, is deleted and 
replaced by a blankjdne of standard length, in administering the test, subjects are 
instructed to fijl in the blanks in the passages, arod their comprehension df these passages 
is indexed by^e percentage of oinitted words thst'^they correcti^ 

The cioze procedure was used in the READNEED research as an alternative to the 
^index of comprehensiorritoalf is obtained by constanicting multiple choice questions about 
e^h passage. The latltar j^rocteSum has two major drawbacks that led to, the decision 
agy^infgh using it. First, the coi^jmiation of multipik-choice questions is highly subjective, 
" and hence both tbe 4«finition^(^ej^ content Tto" be comprehended and the 

nature,^ form, and difficoliy of theq^J^ons maaj be expected to vary greatly depending 



upon the individual preparing the questions. Second, the^^^ maxim uni number of compre- 
hension questiOY»s pnssihlo ior c 50-\vord passage would be far smaller than the 30 items 
provided by the five-cy^cle cio::^ lost of the same material. ' ' ^ 

Research has inaicated that, ; although thoxe is no '^single definitive method for 
measuring reading comprehension, the "mechanical'' clo^e procedure has consistently 
yielded very high correlations with multiple-choice tests and other more subjectively 
constructed measures of comprehension and difficulty (7, 8, 9, 10, 11).' Therefore, the 
weight of the evidence indicates that' the cloze test provides a valid measure of reading 
comprehension- The fact that it is aho strictly objective, and that n independent alternate 
forms can be created .simply by deleting every nth word counting frorp the first, second, 
. . ; or nth word froni\ the beginning of the passage^ further encouraged the use of the 
cloze procedure in the IREADNEED research. T . v ' , 

RELATIONSHIPS OF READING GRADE LEVEL TO V ^ ; 

COIVJPREHENSION OF ARMY^TECHNieAL MATERIALS 

. Bpf pre one can relate different structural properties of a passage to the reading 
^difficulty level of ^the passage, it is necessary to establish the latter through testing 
procedures. Working witlgschool children, previous researchers (3, 6) have specified the 
reading difficulty/levels of a -passage by asserting a criterion, suchj^^s' 75% correct on a 
-niuItipJfirdimce^G^^ test on the passage, and determining the lowest school 

grade at which the average "bonipreHensimr^c<)re--m^ 

for instance, eighth grade— is then taken as the reading grade level for w;hich the passage 
is comprehensible, and the passage is said to be of eighth-grade reading difficulty . 

„ As a modification of this procedure, ^^students , may be tested on a standardiz^ 
reading test, and also on their ability to^cornprehend test passaged. A determination then' 
may be rnade of the lowest measured reading, grade level at whi<::h the average cornpre- 
hension score for the test passages matches thk criterion^n our example, 75% correct 
Using this procedure, it^is possible to say that persons scoring at the seventh-grade level 
on the standardized test score, on the average, 75% correct on the comprehension test. 
The test passage then is said to be of seventh-grade reading difficulty. 

In the cas=^' of the adult Army population, there are doubts about the meaningful- 
ness of the literal interpretation of a ^ade-level ^xpres^ion of reading ability determined 
by grade-school reading tests. -Nevertheless, the expression of the reading grade level norm 
is useful as a roughly common metric for comparing reading ability of an individual and 
the reading-lability, required to ufiderstand a passages-even without reference to school 
grades. For this .reason, the secorid procedure outlined^ was used in developing the 
FORCAST formula. ^ 

On the basis of prior research (8, 9^), the criterion of corhprehension for the 
READNEED exg^erimental passages was established as '35% correct on the cloze test for a 
passage. The referenced research has indicated that, with cloze score of 35% correct, 
one might reasonably expect to obtain about 70% correct on a multiplie-choiGe test built 
for jbhe passage. ^ ^- : ' ' ' 

To determine reading ;grade levels of men , achieving the 35% correct criterion for 
each of the 12 ' experimeintalo passages, use was -^made of the Reading test, U.S^^ Armed 
Forc|5, Institute (USAFI) Achievement Tests III, Abbreviated Edition, Form A (a special " 
printing of the Metropolitan Achievement Test, Advanced Battery). This*test was adminis- 
tered to 395 unselected Army recruits imdefgoing Reception Station processing at Fort 
6rd, California, in February and March of 1971. • . . ' . 

In the testing sessions, the men were first given the' USAFI reading test Immediately 
afterwards, veach man was given one variation of a set of six experimental cloze passages, 
and tested on his cpmprehension of the passages. - ' 



/ 



The 12 experimental passages were divided into two sets of six passages because of 
limitations in testing time and a man's endurance in taking tests. About half (20Q) tHe 
men were administered the six" test passages numbered with the initial digit 1, and the 
other half (195) the test passages numbered with the initial digit 2 (Appendix A), The 
two sets of cloze passages were judmentally equated on difficulty. Each set was prepared 
in each of five -^variations: In the first variation, every fifth word was deleted, starting 
vnth the first word; in the second variation, every fifth word was deleted, starting vnth 
the second word, and so forth, until five variatipns were preparedr By this means, cloze 
scpres were obtained for all words in a passage. Each of the cloze passages was scored as 
the number of correct responses; the maximum for each passage v/as 30. Except for 
minor spelling errors, a response was scored as correct only if it exactly matched, the ' 
delefed word. ' - ' . ' 

The USAFI reading comprehension passages v/ere scored in standard fashion, vnth 
raw scores converted to reading grade level scores having a^ possible range of grades frorn 
1.3 to 12.9. Because different men took different passage sets, and a priori efforts to 
equate '.the two sets of passages, for ceading difficulty were not entirely successful, it is 
important to note that the two groups of men were of equal reading ability. USAFL- 
reading grade level means for the two subsamples of men were 9.40 and .9.42, vnth 
s^&ndard deviations of 2.7 and 2.5, respectively, On this basis, data from the two septate 
sets of passages were pooled into one set of 12 passages. 

With routine testing instructions, testing was completed in a two-hour period.. 



RESULTS 



AFQT AND READING GRADE LEVEL DISTRTBDTlONS 

FOR THE SUBJECT SAMPLE 



^ Table 1 presents the percentage distributions of AFQT and reading level in the 
sample of men tested on the USAFI Reading and cloze test passages. Both show wide 

Table 1 ^ / V 



Percentage Distribution of Subjects by 
AFQT Level and USAFI Reading Grade Level 

,(N = 395) 



AFQT Level 


Percent 


Reading Grade 


Percent , 


10-19 


8.9 


2 


0.8 


20-29 


•10.9 


3 


2.0 


30-39 


17.0- 


4 


4.3 / 


40-49 . 


11.7 


5 


6.6 


50-59 


12.2 


6 


6.6 


60-69 


8.1 


7 


7.1 ' 


70-79 


11.2 


8 


14.2 


80-89 


12.2 


9 


10.3 


90-99 


7.6 


10 


7.8 






11 


25.0 






12 


15.1 


Toial \ 


100 


Total 


100 



10 




distribution r^ges. The wide range of abilities of the sample and liiniU i testing time 
necessitated th^e compromise choice of aji intermediate-level reading ^iv^^Ip with a ceiling 
RGL of 12.9 tlrat was too low to differentiate the moro able readers in the sample. Thus, 
more than 40% of the sample obtained reading scores at the ll£h and 12th grade levels. 
Although" they are fully descriptive of this sample, these distributions are not necessarily 
representative ol general Army input, because of the limited testing time and single 
location (Fort Ord) of the sampling. 

■V ■ 

o 

iNTERCORRELATIOIMS AMONG READING MEASURES AND AFQT 

This study made use of reading measures from two sources, the USAFI Achievement 
Test and cloze tests of the experimental passages. The USAFI test is composed of two 
parts— Redding (paragraph comprehension tested by four alternative multiple-choice tests) 
and Word Knowledge (vocabulary test by the same type of multiple-choice procedure); 
by combining the" two parts, the USAFI Composite score is obtained. 

in the cloze tests, as mentioned previously, a given subject was administered only six 
of the 12 experimehtal cloze test passaiges; therefore, there are two sets of cloze tests, 
series 11-16 and 21-26. .Table 2 presents the intercdrrelations among these measures and 
^so AFQT. The relationship between USAFI Reading and the cloze total for each of the 
two sets of six passages, .83 and .75, is sufficiently high to indicate appreciable 
correspondence between these two reading-comprehension measures. USAFI Word- 
Knowledge and Reading scores are highly related to each other and show almost identicsd 
relationships to the other variables. The composite score affords little gain over either of 
its components. The consistent relationship (rs ranging from .68 to .72) between AFQT 
and the four reading measures reflects the large reading component of the AFQT 
(one-^[uarter vocabulary and one-quarter arithiriietic word problems). 



Table 2 

Intercorrelations Among 
Reading Measures and AFQT 



Reading Measure 




2 


3 


4 


5 


6 


1 .USAF]=I Reading ^ ... . 

2 .USAIj=l Word Knowledge " 




.85 


.96 

.96- 


.83 
.85 


-.75 
.75 


.68 

.68 



3 USAFI Composite 

4 Cloze (IMS) 

5 Cloze (21*26) 

6 AFQT 



.87 .78 



.71 
.72 
.68 



The most significant aspect of the data in Table 2 is that the high correlations 
among the cloze tests and. USAFI Reading test support the previous statement that the 
cloze test procedure produces at least as valid a measure of comprehension as the typical 
multiple-choice test procedure. " . 
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CLOZE TEST RESULTS 

The means and standard dev-iations (SD) of the cloze percent-correct sroro^ for th« 

SbLTTheseT^"' " °' ^"^^^"^^ ^'"''^^'y' are shr^ Trow^S ^^^^^^^ 

rable 3. These scores are expressed as the percentage of correct answers eiven to the -^O 
cloze Items on each of the 150-word experimentel passages. Gi^nThTaLTleve l an^^^ 
techmcal nature of the reading passages, the range%f1hese ^eLs 0%)^^^^^^^ 
ponds we l wi«i the typical findingi generated by this^ measx^e when it is apphed to 
re2?n,at'*-f'" """^f^^ ^^'^^^^^ ^^^^ ^^^e simplesf element^ 'chool 

Sr^'froT;::^^^^^^^^ The variability (SD) among subjects tested is notably 

Table 3 v 

Readabmty Levels by Different Measures and 
Cloze Percent-Correct Scores for Individual Passages 



Property 



Passages 



21 



12 



1. Scaled RGL 

2. FOR CAST 

Readability level 

3. Flesch 

Readability Level 

4. Oale-Chall 

Readability Level 

5. Cloze 

Mean Score ' 

6. cloze 

SD (%) , /' 



11 



13 23 22 15 



16 



26 



25 



24 



14 



6.0 7.0 7.0 7.3 .9.1 9.6 11.4 11.8 12.0 12.0 12.1 13.0 

S.G 7.8 7.6 9.4 10.1 10.7 12.1. 13.2 12.2 13.2 11.3 10.9 

^ ^ 5 7 13-16 10-12 13-16 16+ 16+16+ 13-16 13.16 

7-8 7-8 5-6 7-8 11-12 9-10 13-15 16+ 16+ 16+ 13-15 13-15^ 

54.0 46.7 45.1 45.7 35.1 33.5 27.3 25.4 25.0 23' 1 233 19.4 

15.9 17.9 17.7/ 17.4 14.4 14.3 14.4 15.0 16.0 14.3 13.8 14.4 



these^Ssafes^bv^'r^^^^^^^ ?h?f 'ff^^-^-^-'^-^-^ (R^L) score that was assigned to' 
cnese passages by use of the cloze dnterion of 35% correct (Row 1) In readahilitv 
• research usmg multiple-choice items/to measure passage conipiSielion it h^ be^^^^ 

SStv of\?" ^ Percentage./depending as it does upon the generally ; unknown 
difficulty of the multiple-choice qWstions, was initially arbitrary but has tended in 
become conventional. Comparative studies "(8, 9) indicate , that^^loze s^ore of 40% 
tesrof'°"''''T^''-'^Pr'u"^^^^' to this criterion of 75% -correct on mS^^^^ 
?r^£rion ofTsfrrt^c^ the/preseht research we have adopted thMomewhat 1^^^^^^^ 
SS\SL2pS to a^^^^^^^^ '^"^^^ as our criterion plpdssage comprehension." 

o^^tLT^ f ^ , S ^ , "^^ correct on a mul£iple-choice test of comprehension a 

co^oreL^^^^^^ Using ^hi« criterion, the rSabiSty o? 

comprehensibihty of a. passage, expressed in terms of reading grade level YrgL) \Z 
'^^eA.^the ^u^es^reading grade level in u>hich 50% olfhTmen ream^lt Zt 



/ 
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who read at the 6.0 level, as meafviired by the USAFI Achievement test, achieved aD least 
35%^ correct on the cloze test for passage ,21. A similar interpretation holds for the 
remaining passages. ' 

Data on which the deterniination of scaled RGL for each passage in Row 1 of Talkie 
3' is based are contained in Appendix C. To^ permit examination of the con5g£}trences of 
adopting a different criterion level of compreTiension, parallel dataTaie given there for the 
30, 35, 40, and 45% correct levels of comprehension. ^ 

ijUthdugh the procedure for constructing a^ cloze test (replacing every fifth woVd 
with blank) is ceriainly objective, the difficulty of a test so constructed .may vary as a 
function of which starting place is selected--and thus, which fifth of the words are 
deleted. Because of this, in the present research, each set of six experimental cloze tests 
was administered in five variations. In the first variation, every fifth word, starting with 
the first word, ,was deleted^ in the second variation every fifth word, starting with the 
second word was deleted, and so forth. Thus, every word in every passage served as a 
cloze test item for one-fifth of the subjects. Mean percent correct cloze scoires for each of 
the five variations are shown in Appendix D. . • x(/ . '\ 

For many passages,, the variability of the mean' cloze score fr'bm. variation to 
variation is substantial. Since only about 40 men were tested on any one variation of a 
set of six passages, . it is likely that* differences in reading ability among men randomly 
receiving different variations, as well as differences in cloze test difficulty due to the 
variations, contribute to the differences in cloze scores among the variations shown in 
'Appendix D.' - ; . - 

In the following description of the development of the FORCAST readability 
formula, cloze scores were computed; by summing over all versions of the' cloze tests for 
each passage. 

CONSTRUCTION OF THE FORCAST READABILITY llsbEX 

To this point, the steps taken to construct a job-related readability index have been 
described: \ . . ^ ' 

(1) Determination ofyjobs to. be included. 

(2) Determination of essential job reading materials. ■ 

(3) Selection and assessment^of reading pEtssag^^^ the job leading materials 
in terms of their' appro properties. . 

. (4) Measurement of in^dividual reader's comprehension of passages using the job 
reading materials. 

C5)^ Scaling^ of^. passages by the reading grade level (RGL) required for a 
designated criterion level of comprehension. 

A final "^step involved determining optimal weights of the structural properties of 
passages to enter into a formula to maximize the prediction of the RGL required to 
comprehend a passage at the designated criterion levjel. Table 4 presents the inte^orrela- 
tions among the 15 variables characterizing the structure of the experimented passa^fes^and 
their relationship to the mefe" cloze scores on these" passages (summed over all reading 
grade leyels^ and^yover-- iall variations! of the cloze tests for each ; passage). While the 
interrelations among -the various structural properties are of some interest, because they 
suggest the exteili to' which a structural property covaries v^rith another like property and 
may be substituted for it in a readability formida, major interest is in the relationship of 
each of the structural properties to the^ cloze score (Column 16 of Table 4). 

Of the several structural variables showing, hi^ correlation with the cloze score, the 
number of one-syllable words is preferred for the KORCAST readability index. Not only 
it b^n found useful in othar, general-purpose' readability indices, it is also the easiest 
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measure to apply. Moreover, the relationship is sufficiently high (.86) to permit its Use in 
a simple, single-variable readability formula. 

In order to determine whether combinations of these 15 structural variables might 
relate more closely to meah cloze score than did any single variable, selected sets of two ' 
and three .variables were examined by the multiple-linear-regression technique. As shown 
.in Appendix E, this approach provided no practical advantage and was discontinued. In 
view of the high redundancy among the predictors, the outcome was not surprising. 

The correlation between the one-syllable words and cloze scores permits the estima- 
tion of a cloze score for a given passage based upon the number of one-syllable words in 
the passage. While such an estimate may be of interest for some purposes, it is more 
generally desired that the reading difficulty of a passage be stated in terms of the RGL of 
difficulty of the material. Accordingly, the correlations between one-syllable words and 
the RGL associated with the clo;ze performance criterion (35% correct) were calculated 
for each passage as given in Table 3. This correlation was .87. 

Regression analysis produced the follovring preliminary readability formula: 

RGL = 20.43 — (.11) (number of one-syllable words) (l) 

The values 20.43 and .11 were reduced to 20 and .10,^ and .10 was^changed to 1/10, in 
order to produce the very si;nple readability formula dubbed FORCAST (FORd, CAylor, 
^Ticht). The FORCAST formula is: 

FORCAST readability in RGL = 20 - ^^^^^rof one-syllable words 

To use the FORCAST formula to predict the RGL readability of a 150-word 
passage, one (a) counts the number of one-syllable words in the ISO-word passage, 
(b) divides that number by 10, and (c) subtracts that value from 20. 

For example, the estimateeed RGL of readability of a 150-word passage containing 
96 one-syllable words wotUd be 20 - 9.6 = 10.4, or about the middle of the lOth-grade 
level. This corresponds to the prediction that, on the average, men reading at the grade 
10.4 level would ,be expected to get 35% correct on a five-cycle cloze test for the 
passage. 



APPLYING AND EVALUATING THE FORMULA 

LIMITATIONSf TO THE FORCAST READABILITY FORMULA 

^ . The. FORCAST formula -was developed for and on a defined body of reading 

material (Army techriicapptureading matter) and a defined population of readers (young 
male soldiers). Unlike q^st general-purpose readability formulas, it was pot intended for 
use .with^lementary and secondary\chool materials, or with newspaper and magazines, 
and .its applicability to these IS not demonstrated. ^ 

One apparent limitation to the FORCAST index is its ^restxicted "range. In the 
- unlikely limiting case that all words in a 150-word passage of ;jol3 material should be 
monosyllabic, th^ readability of the. passage would be indexed as fifth grade (5.0) and the 
, - ' iiidex will go no lower* To date, no passage§c of such low readability have been 
encountered in * Army job material and a diligent search was necessary to turn up . a ' 





passage as low as the 6.0 scaled reading grade level passage used in the experimental list. 
At the other extreme, the fact that a maximum score on the USAFI scale used in this 
study was^normed as RGL 12.9 set this as the upper limit that could be assigned to the 
readability' of a passage in developing the FORCAST formula. FORC AST predictions 
above that point are based upon linear extrapolation. ; How^ever, any reading measure 
appropriate to the wide range of soldiers' reading ability must similarly be based upon 
extrapolation at both extremes. Any passage characterized as the 12th grade readability is 
a difficult one, and there is little need in practical application for precision beyond the 
simple ordering of even more difficult passages. 



RELATIONSHIP AMONG FORCAST , FLESCH, AND 
DALE-CHALL READABILITY MEASURES 

Table 5 presents intercorrelations among FORCAST, Flesch, and Dale-Chali reada- 
bility indices and scaled reading grade level (RGL) scores for the experimental passages. 
From the high intercorrelations among the three readability indices, it is apparent that 
they are functioning in a highly parallel -fashion in ordering the experimental passages on 
readability. Similarly, they are all highly, and about equally, related to the scaled RGL 
required to comprehend the passages. ^. J 

Table 5 

Development of the FORCAST Formula: Means and 
Intercorrelations of Four Indexes of Passage Difficulty 



lnde;< 


Intercorrelation 


Mean 


SD 


1 


2 


3 


4 


1 FORCAST 




.92 


^94 


.87. 


10.6 


1.9 


2 Flesch 


.92 




.97 


.92 


11.8 


'4.4 


3 Dale-Chaii 


.94 


.97 




.93 


11.6 


3.9 


4 Scaled RGL 


\87 


.92 


.93 




9.9 


215 
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While the Flesch and Dale-Chall forfhiilas, developed on general educational material 
*and readers, show high validity in the present situation, because of the simplicity of the 
FORCAST formula,, it is a more desirable, readability formula for use whenever a^ult 
technical materials must be evaluated by relatively unsophisticated clerical personnel. 

CROSS-VALIDATION 

A cross- validation study was conducted, in order to determine the validity of the 
FORCAST readability index for a sample of Army job reading material independent, of 
that on which it was empirically derived. The initial design was replicated, using another 
. sample of 12 Army job reading passages from the same MOSs and another sample of 365 
. Army recruits at the Fort Ord Reception Station. Passages ranged from RGL 7.0. to 12.7 
as indexed by the FORCAST formula. 

As indicated in Table 6, the FORCAST values for the 12 passages correlated .77 



grade level at which 50% of the men at that level made a score of at -least 35% correct on 
the cloze test of the passage. Mean FORCiA^ST and scaled RGL scores for the 12 passages' 
were 9.4 and 10.4, respectively. 

Table 6 ... 



Cross-Valldation of the FORCAST Formula: Means and 
Intercorrelations Amc g Four Indexiss of Passage Difficulty 



Index 


i ercorrelation 


Mean 


SD 


1 


2 


3 


4 


1 FORCAST 




.98 


.95. 


.77 


9.4 


: 2.0 


2 Flesch 


.98 




.94 : 


.78 


9.4 > 


4.2 


3 Daie-Chall 


.95 . 


.94 




.86 


9.5 


4.0 


4 Scaled RGL 


.77 


.78 


.86 




10.4 


2.2 



The generally high rs between the -FORCAST and the Flesch and Dale-Chall 
. formulas were again found. With this new set of passages, the latter formulas were more 

accurate in estimating the mean of the scaled RGL. In general, the results of | this 
* cross-validation are, within normal sampling fluctuation limits, fully consonant with' the 

basic findings and warrant the use of the FORCAST index for its intended purpose. 

USING THE FORCAST READABILITY FORMULA WITH 
ARMY TECHNICAL MATERIAL 

The FORCAST index was' developed to provide, an estimate of the reading ability 
level required to read and understand Army job reading material.\It can be applied to a 
single passage, a sample of material from an MOS, or the entire body of reading material 
in an MOS. To obtain«an estimate of the reading grade level of difficulty of materials in 
the seven MOSs from which the READNEEO experimental passages were obtained, the 
formula was applied to aiLthe materials sampled in each MOS. These materials represent 
the reading materials that a man must study to pass his MOS proficiency test (unless he 
can learn the information in some other way); these_are, then, critical job reading 
materials. The manuals and regiUations containing the prescribed job reading material, as 
well as the number of s^ples taken of each publication, are listed in Appendix F. 

Table 7 presents, for each MOS, the cumulative percentage of job reading materials 
for each of seven grade levels of difficulty, estimated by the FORCAST formula. Using 
the reading grade level 9 to 9.9 as the best estimate of the average reading ability range 
of the general Army population (see Table 1 and reference 12), these seven MOSs can be 
rahkied on how well the reading difficulty of materials in the MOS matches the average 
reading ability of Army personnel (i.e., 9 to 9.9). Applying this procedure, we find that 
. the Medical Specialist (91B20) has the largest proportion (24.4%) of materials written dt 
or below the 9.9 reading level, and is the least demanding of readings skills. The remaining 
MOSs have the following percentage of material wi'itten at the~ 9.9 level: Light Weapons 
liifantryman, 18.3%; Military Policeman, 15:1%; General.. Vehicle Repairman, 13:4%; 
Arinorer/Unit Supply Specialist, 10,8%; Ground Control Radar Repairman, ' 4.2%; 
Personnel Specialist, 2.2%. 



Table 7 



Cumulative Percemtage Distribution of Job Readirrg Materials for 
Seven MOSs and Seven FORCAST Readabil ty Levels 





Military Occupational Specialty^ 


RGL^ 
















11620 


26320 


63B20 


71H20 


76Y20 


91B20 


95B20 




(N ^ 104) 


(N 95) 


(N - 108) 


(N -95) 


(N - 83) 


(N - 90) 


(N - 138) 



6- 6.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 

7- 7.9 1.0 ^ 0.0 0.0 0.0 0.0 0.0 0,7 ^ 

8- 8.9 4.8 T.O " 3.3 1.1 3.6 2.2 5.0 * 

9- 9.9 18(3 ^.2 13.4 2.2 10.8 24.4 , 15.1 

10- 10.9 41.4 9.5 . 36.3 3.3 20.4 47.8 34.0 

11- 11.9 71.2-^ 42.1 61.8 37.0 57.6 77.8 62.2 
12.0+ 100.0 100.0 100.0 100.0 100.0 100. 0 100.^ 



^RGL—Reading Grade Level of difficulty of job printed materiafs determined by the FORCAST 
fcimula. 

h 1 B20, Light Weapons Infantryman; 26D20, Ground Control Radar Repairman; 63B20, Wheel* 
Vehicle Repairman; 71 H20, Personnel Specialist; 76Y20, Armorer/U/iii Supply Specialist; 91 B20, Medical - 
Specialist; 95B20, Military Policeman. / 

- \ 

Although there are clear differences in readability of job printed materials among 
the MOSs, all. the MOSs show readability levels well above the ni'nth-grade level. These ; 
findings confirm previous observations (1^!4), aVid once again suggest that quality-control 
prociedures should be applied to printed "materials in order to make them more useful to. 
the majority of personnel. » * i - 




Chapter 3 

j i REAiL'NG REQUIREMENTS ESTIMATED FROM 
T '-r PRIMARY MOS ENLISTED EVALUATION TEST 

r-^EMENTS IN ESTIMATING READING REQUIREMEKTS 

A pro L^a_.^e using iata from current Army data banks for estimating functional 
literacy req-nreinents of Army MOSs is discussed in this chapter. Functional literacy is 
defined here as ftiat level of reading ability that is minimally sufficient for satisfactory 
job perforiT ance. In principle, the task is straightforward: To determine the literacy 
requirement- of a job, one needs a sample of job incumbents, a measure of their reading 
ability, and a measure of their job proficiency— along with an authoritative definition of 
the level of job performance that is "satisfactory." In view of the typical, positive 
relationship between literacy and job 'knowledge (1), the job reading requirement can be 
readily determined, through correlational analyses, as the minimum literacy level asso- 
^ ciated with^ "satisfactory" job performance. 

Given the minimum necessary data— reading scores, job-proficiency scores, and/ a 
decision on what constitutes a satisfactory level of job proficiency — it is possible to apply 
this procedure to any MOS. 



READING SCORES . , 

Except for some Project 100,000 men, there are no direct measures of reading 
ability in Arjny personnel records. However, there is consistent evidence of the sub- 
stantial relationship between AFQT and reading grade level. For approximately 1,500 job 
incumbents of widely varying job experience in four MOSs, the AFQT-stand'ardized 
reading test correlation was .68 (j^), and for three 200-man groups of unselected recruits^ 
in a Reception Station, median r was .79 (1), For men entering Combat Support Training 
(CST) in three MOSs, the median r was .75, as it was for three samples of about 200 
men each upon completion of CST (Chapter 4). An of .68 was found in another 
Reception Station sample of 395 recruits; in this case, a different standardized reading 
test was used (Chapter 2). 

The stability and level of this relationship, in conjunction with the consistent 
linearity of the regression, indicate that reading scores, although not ditectly available, 
can be meaningfully estimated from AFQT. Admittedly, the AFQT-based estimates of 
reading ability are imprecise, but. the loss of precision is unavoidable in any case of 
regression-based estimates. If such a procedure as described herein were to be undertaken 
operationally, it probably would be because of the ready availability of the AFQT in 
personnel records, and the desire tb avoid the expense of wide-scale testing of reading- 
skill levels— a trade-off of precision for cost savings and convenience. 
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profi<?iency in his MOS. The MOS/ET (based .upon multip! -choice, job-knowledge items- 
submitted by the proponent agency for tlie MOS) is centr-Uy revised, administered, and 
scored by the Army Enhsted Evaluation Center. The score on the .l25-item test is 
available as an independent entity. Its identity, however, is lost when it is merged in 
weighted combination with the Enlisted Efficiency Report EER) rating and the resulting 
composite score is ncrmed and recorded in the individual's files for use in MOS 
verification. Specialty and Superior Performance Pay qualification, and other personnel 
actions. The MOS/ET score is taken as the most authoritative and objective index of job 
proficiency, available in existing Army administrative records. 

CRITERION LEVEL OF JOB PERFORMANCE 

The empirical answer to the question, "How well must a m,an read to do the job?" 
depends upon two factors: (a) the empirical relationship between reading and job pro- 
ficiency in that MOS, and (b) the judgmental decision as to how much proficiency it 
takes to "do the job." 

At present, criterion scores defining adequate proficiency on the MOS/ETs are set 
by the proponent agencies for each MOS, in conference with the Enlisted Evaluation 
Center. Such criterion scores are based jointly on a consideration of the distribution of 
proficiency scores- in the MOS/ET, the best judgment of the proponent agency as to its 
manpower needs, and experience-based expectations about the likelihood of successful 
job ^performance by men scoring below the criterion score. - 

Whatever the bases for setting criterion scores,'' it is sufficient for the READNEED 
research that such criteria are in existence, and thus we have the elements in existing, 
standard records for estimating reading level requirements of any Army MOS— AFQTs, 
from which reading ability, may be estimated, and an objective, administratively sanc- 
tioned measure of job proficiency to serve as a criterion to which reading ability may be 
related.^ ' 

PROCEDURE 

The general' procedure for estimating the- reading demands of a given MOS consists 
of six steps: • 

(1) Select the MOSs to be'studied. 

(2) Select a sample of men carrying that MOS. 

(3) Extract AFQT, and MOS/ET scores from existing records for each member 
of the sample. 

(4) Compute the mean MOS/ET scores for men at each AFQT decile. 

* Although these data are available, they are not to be found in a common source. Scores oh the 
MOS/ET are available in the records of the Enlisted Evaluation Center (EEC); AFQT scores for all EM 
are stored on PERSINCOM's Enlisted Master Tape Records (EMTR) and subsidiary tapes as well as in 
the 201 Files for these men at the Enlisted Personnel Support Center (AFPERCEN). Although the 
PERSINCOM tapes contain the PMOS Enlisted Evaluation Score > and the AFQT, the MOS/ET and 
Enlisted Efficiency Rating (EER) composite, virhicK.is the Enlisted Evaluation Score, is not considered 
an appropriate criterion measure for this research because data (1) in'dicate tliat the rating component is 
* not likely to be substantially related £o job knowledge. Conceptually, the- procedure described remains 
unaltered by the lack of a common source for AFQT and MOS/ET scores; in practice one step is addefl. 
Since data are in ADP format at both .PERSINCOM and EEC, the sample for men in a given MOS can 
be selected from either source and, through matching Social Security numbers, the second datum can he 
obtained from the other source. Such'^is the procedure that has been followed in the present work! 



(5) Ascertain tha lowest AFQT level at.vvhich the proponent-based. .MOS/KT 
criterion level is met. 

(6) Convert that AFQT level to a reading gracie level equivalent using the 
standard regression equation provided in this report. 

These steps will be described as they were, accomplished in the present research. 

SELECTING MOSs ' 

Three criteria were used in selecting the MOSs whose reading level requirements 
were studied. They were ref)resentative of a wide range of job families and job require- 
ments, they were linked with other READNEED and REALISTIC research on MOS 
reading requirements, and data were available on a timely basis. 
The following MOSs were chosen for study: 
11B20 Light Weapons Infantryman 
31E20 Field Radio Repairman 
63B20 Wheel Vehicle Mechanic 
71H20 Personnel Specialist " 
. 76Y20 Armor/Unit Supply Specialist 

91B20 Medical Specialist ' . 

95B20 Mihtary Policeman . 

SELECTING PERSONNEL SAMPLE AND OBTAINING SCORES 

. For each of the seven MOSs, a sizable sample of MOS/ET data was obtained, and 
for each MOS, the Enlisted ^Evaluation Center (EEC) provided the following data for a 
samjple of 400 men: 

• Name ^ 

• Social Security Account Number (SSAN) 

• Pay Grade 

^ • Enlisted Evaluation Test Score (MOS/ET) 

• Enlisted Efficiency Report rating score (EER) 

• iClinimum Passing EET Score established by the proponent agency for the MOS 
From these samples, a subset of 100 men was selected for each MOS in order to; 

provide a full range of MOS/ET scores with a heavy concentration of subjects m the 
region of the minimum passing score on the MOS/ET/ established by the .proponent 
agency for that MOS, AFQT scores for these men were extracted from their 201 files at 
the 'U.S. Personnel Services Support Center (AGPERCEN).^ Between ^the varying dates of 
MOS/ET testing an the different MOSs and the time the READNEED* research assistant 
entered the AGPERCEN files, a considerable interval had elapsed. To varying degrees in 
the different MOSs, 201 files of men who were selected for study were no longer 
available because, in the interim these men had complet&d their term of active duty. 
.Subjects whose 201 files were not available were replaced from the 400-niah rosters by 
substitutes with MOS/ET scores as nearly equivalent as possible. In the Wheel Vehicle 
Mechanic MOS, in particular, there were insufficient low-scoring MOS/ET substitutes 
available from the 400-man sample to provide the intended distribution of MOS/ET 
scores. This somewhat tortuous try out procedure resulted in a set of approximately 100 
subjects in each of the MOSs who were distributed on AFQT, EER (Enlisted Efficiency 
*' ■ . ' 

' Time limits precluded the less cumbersome procedure of extracting AFQT . scores from the 
PERSINCOM tapes. This wpuld be the recommended procedure in a»,i operational program. 



Rating), and MOS/ET as shown in Table 8. The minimum passing MOS/ET score set by 
the proponent agencies for these MOSs is also shown in this t^ble. 

Tables ^ 
Percentage Distributions of AFQT, EER, MOS/ET, and MPRTS Scores 



Minimum 
passing score 
(MPRTS) set 
by proponent 

agency 

^ - 



43 



60 



42 



57 



59 











MOS - 








Test 


1 1B 
1 nf antrv 
(N-107) 


31 E 
Electronics 
(N =98) 


63B 
Mechanic 
(N = 1O0) 


71 H 
Personnel 
(N = 101) 


76 Y 
Supply 
■ (N =.98) 


Medical 
(N ^ 99) 

^ ' 


95B 
MP 
IN = 100) 


AFQT 
















91-99 i 


5 . 


, 20 


- 1 


' 9 


4 


5 


1 


Q1 on 

o 1 -y u 


Q 


on 


4 


1 0 


4 


y 


1 / 


71-80 


. 7 


11 


' 6 


16 ' 


2 


8 


21 


61-70 


5 


8 


7 


12 


9 


6 


11 


51-60 


5 


8 


11 


' 8 


Q 
0 


15 


14 


41-50 


15 


11. 


13 


14 


1 1 ' 


15 


9 


31-40 


21 


7 


28 


12 


^0 




14 


21-30 


19 


q 


1 1 


5 


1 R 


Q 


f5 


11-20 


20 


6 


19 


9 


.21 


5 


1 . 


^ EER 
















91-125 


78 


90 


69 ^ 


85 


73 


84 


80 


81-90 


9 


8 


10 


7 


7 


7 


8 


71-80 


5 


1 


6 


2 


8 


• 6 


9 


61-7tf 


2 


1 


7 


1 


6 


1 




51-60 


1 




4 


2 


•4 


1 


2 


41-50 






3 ' 


2 








0-40 


5 




1 


1 . 


2 


1 


1 


MOS/ET 
















91-125 




8 


2 




2 


5 




81-90 




19 


7 


3 


11 


14 


11 


71-80 


14 


13 


22 


16 


18 


18 


28 


6t"70 


}T 


15 


31 


20 


23 


26 


25 


51-60 


A7 


17 


14 


33 


18 


20 


12 


41-50 V 18 ' 
O40 N34 


10 
17 


15 
9 


17 
11 


16 
12 


12 

5 


18 
6 



65 



46 



Differences between MOSs in the proportions of men at a given MOS/ET score level 
reflect only a decision to oversample in the region of the minimum passing test score. 
Beyond showing a wide range of MOS/ET scores with greatly fluctuating sampling density 
in each MOS, .these MOS/ET distributions have no further independent meaning. 
Simil^ly, the AFQT distributions do not represent random samples of that variable in 
these MOSs and are presented only to show the range and relative frequency of AFQT 
scores in these present data. ' » ^ . 



, Correlational analysis indicated that the relationship between AFQT and EER ratings 
was trivial,: median r for the seven MOSs being .05. This, in conjunction mth the previous 
findings of inconsequential relationships between EER and AFQT, reading, job knowl- 
edge, or job sample? performance (1) led to dropping the .EER ratings from further 
analysis. While the place of the EER in indexing an essential component of total job 
proficiency in the work setting is recognized, it shows no useful relationship to literacy 
or to the measurable comi^onents of job proficiency mediated by reading. 

RELATiNG MOS/ET TO At^QT AND READING ABILITY 

^ The procedure for estimating the reading requirements of MOSs consists of deter- 
mining the lowest AFQT level at. which job incumbents do, on the average, attain a 
specified criterion level on the MOS/ET, and of estimating the reading m-ade ifor that level 
of AFQT. 

.To ascertain -how a|OS/ET performance- varied with AFQT, mean MOS/ET scores 
'vvere computed for men ^at each decile of AFQT (Table 9). These data were then 
smoothed by computing weighted^ MOS/ET means for adjacent AFQT deciles, and a least 
squares best linear fit was applied to the resulting values in each MOS. These data are 
shown separately by MOS in Figuxe.l. For simultaneous comparison, the linear regression 
lines for- all MOSs are brought together .in Figure 2. It is apparent, for these seven MOSs, 
that, to varying but substantial degrees, MOS/ET* performance does increase as a positive 
function of AFQT. 

/ ■ ; 

V Table 9 ' 

Mean MOS/ET Scores by AFQT Decile 







AFQT 


11 B 

Infantry 


31 E 
Electronics 


63B 
M-sJchanic 


711-1 
Personnel 


76Y 
Supply 


91B 

MedlcaJ 


95B 
MP 


11-20 


39.9 


55.3 ^ 


56.8 


56J 


46.4 


. 67.4 


49.0 


21-30' 


44.8 


46.8 


60.6 


32.4 


.52.0 


68.3 


66.3 


31-40 


47.5 


47.7 


57.5. 


50.2 . 


62.2 


60.1 


58.5 


41-50 


. 51.6 


56.2 


67.5 


55.6 


63.4 


59.9 


51.9 


51-60 

61-.70 


47.8 
59.2 


57.5 

62,8' 


• 62.4 
.76,9 


63.5 
^ 57.3 


71.4 
72.8 


65.8 L 64.9 
68.7 ^^-SCT" ' 


71-80 


68.7 


61.1 


71.0 


57.2 


80.5 


73,2 


76.1 


81-90 


71.7 


71.2 


75.8 


67.4 


83.8 


71.4 


66.4 


91-99 


74.0 


77.6 


69.0 


69.0 


83.2 


78,2 ^ 


83.0 



Shown immediately below the AFQT level designations in Figure 2 are the reading 
levels associated with those levels of AFQT. Reading grade levels were estimated from the' 
AFQT on CQie basi? of a sampl^ of 393 unselected recruits at the Fort Ord Reception 
Station \in which the correlation between AFQT and RGL scores on tfie USAFI 
Achievenient Ter.t^ III (Abbreviated Edition) Form A was .68 "(see Chapter 2). The RGLs 
estimated from tJhe AFQT/ by the regression equation ^ [estimated USAFI = .075 
(AFQT) + 5.52] are shown in Figure 3. 
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Regression of USAFI Reading Grade Level on AFQT 
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Figure 3 . 

From Figure 2, the estimated reading level requirements for the seven MOSs can be 
read off directly— provided that one has a criterion level of an MOS/ET score defining 
satisfactory performance. 

As mentioned earlier, in present practice the proponent agency for each MOS 
defines such a criterion by setting a minimum passing MOS/ET score as one requirement 
for an incumbent^s maintaining his MOS classification. For MOS 11B20, Light Weapons 
Infantryman, the MOS/ET criterion level for 1971 was set at 43. In looking at Figure 2 
it can be seen that by enteringr the MOS/ET ordinate at 43, then moving right- until 
encountering the MOS 11B20 line, aiid then moving vertically down to the abscissa, axi 
average MOS/ET score of 43 is obtained at an AFQT score of 26. By interpolation of the 
bottom line of Figure 2 (or from Figure 3) one can see that for AFQT 26, the, 
corresponding reading grade level is 7.5. Thus, the reading requirement for MOS IIB is 
determined as being at the seventh- to eighth-grade level of reading ability. 

Each MOS has its own Enlisted Evaluation Test and its own minimum passing score 
established by its proponent agency. Table 10 shows the minimum passing test scores for 
the seven MOSs and their reading-grade-level requirements determined as above/ For 
MOSs 63B and 95B (Wheel Vehicle Mechanic and Military Policeman), minimum passing 
MOS/ET scores are set so low that it precludes determining the reading requirement of 
; the MOSs any more precisely than below the seventh-grade reading level. For the other 
MOSs, the reading. requirement ranges from 7.5 to 9-9. Men reading below the reading 
level requirement for their MOS do not, on the average, m'eet this minimum-criterion level 
of job proficiency on the mandatory MOS/ET. 



THE EFFECTS OF INCREASING THE CRITERION LEVEL 

The minimum passing scores are set for reasons of personnel administration, and 
reflect retention policies determined, in part, by supply and demand considerations. In 
view ot the fact that on these 125-item, four-alternative, multiple-choice tests random 
guessing would yield an! average score of 31.25, 'the minimum passing criterion in some 
, MOSs is truly minimal. Table 10 illustrates the effects of increasing the MOS/ET criterion 



Table 10 . r " 

MOS Reading Requirements Under Various Criteria 









MOS/ET, Items Correct 


MOS 


MPTS^ 


RGL° 








60 


65 


70 


Infantry, 












11B 


43 


7.5 


10.3 


1 1.1 


12.0 


ICV' Li \JI Hub, 








r 




31E 


60 


9.8 


9.8 


10.8 


11.7 


Mechanic, 












63B ' 


42 


c 


7.6 


9.4 


; 11.1 


Personnel, 












71H 


^57 


9.8 


10.6 


11.9 


13.0 


Suppjy, 












76Y 


59 


.8.1 


8.3 > 


9.2 


^^10.0 


Medical, 










91 B 


65 


8.6 


6.6 


8.6 


. 10.7 


MP, 












95B 


46 ' 


c 


8.5 


9.9 


11.3 



^Minimum passing MOS/ET score defined by proponent agency. 
^Reading Grade Level. 
^Below 7.0 reading grade level. 



on the estimated reading requirements. Data are presented for criterion scored of 60, 65, 
and 70 items correct on the MOS/ET. . ' ■ ' ^ 

Considering that these criterion levels respectively represent 48, 52, and 56% of the 
maximum possible score on the MOS/ET, the reading- level requirements rise sharply in 
these MOSs for these nd\ninally small increments in proficiency definitiorv^ Not only are 
these tests measuring with great sensitivity in this middle range of possible scores, there is 
also the indication that a substantial portion of MOS/ET job-knowledge items are rarely^ 
passed. ' ^ r 

At the 60- items-correct criterion level, literacy requirements differ considerably over 
MQSs, ranging from reading grade levels 6.8 to 10.5. Diffierences between MOSs in 
literacy reguirements directly express differences in AFQT, and indirectly express differ- 
'ences in estimated reading grade level at which incumbents attain 48% of maximum on 
these tests of MOS job knowledge. At higher criterion, levels, job reading requirements 
increase and differences between MOSiS shrink as the effective celling -is approached. 

The level of reading required for an MOS reflects the criterion lever established as 
sufficient by management. Generally speaking, the higher this criterion of proficiency, the 
more demanding the requirement for literacy. Therq is, then, no single unitary level of 
reading skill to be designated as the job reading level requirement. "Rather, there are as 
Riany 'levels of reading requirements as there are leVels of job proficiency; the determina- 
tion- (if the job-proficiency level deemed sufxicient is not the proper province- of the 
researcher, but is rather a decision to be made by the responsible management. ^: 

For the' above reasons, the entire procedure described in this 'chapter rests squarely 
on the MOS/ET as Army 'mianagement's choice of measures of job-relevant and job- 
essential knowledge. This positipn is. buttressed by the Army's use of the MOS/ET scpre . 
as a a sufficient measure in- itself for invalidating an MOS classification and, in conjunction 
with the EER, as the sufficient measure for selective awarding of proficiency pay. 
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The purpose of establishing the job reading requirement for an MOS is to determine 
how well men must read in order to do their job with a desired level of proficiency, 
particularly when the desired level of proficiency increases beyond that of the break-in 
apprenticeship period. Given the present and probable literacy levels oi^'^Mmy input in 
the future, it does not seem likely that MOSs can be fully manned by men already 
possessing the literacy skill levels required by these MOSs for satisfactory job" perform- 
ance. Rather, the establishment of MOS re^ading level requirements below which job 
proficiency is inadequate serves to specify existing objectives, which, to the extent that 
they cannot be met by selection and ''classification, constitute training goals iagainst which 
tKe effectiveness oS29literaey and job-training programs can be gauged. In this regard, it 
-should • be noted that fpr those five MOSs for which data of Figure 2 permit a 
detO;minate solution, the MOS reading requirement ranges from 2.5 to 4.9 reading grade 
' levels above the fifth-grade graduation criterion of the current Army remedial reading 
program! These data (and others, 1) indicate that such training should minimally aim at 
producing seventh-grade reading ability.^ 
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,^Work to produce a prototype Army literacy-training program targeted at producing grade 7.0 
rfcadiag skill. level is under way in^HuniRRO Work Unit FLIT°' ^ . * ' 
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Chapter 4 \ . 

DEVELOPMENT, STANDARDIZATION, AND VALIDATION OF 
JOB READING TASK TESTS FOR THREE MOSs \ 

The methods for establishing literacy requirements of Army MOSs which were 
described in Chapters 2 and 3, are relatively simple and inexpensive, but they do not 
involve the direct assessment of the ability to read and understand job reading materials. 
The readability approach of Chapter 2 estimates the ease of learning from or of utilizing 
job reading materials .on the basis of a count of simpFe structural properties of the text. 
The use of existing Ai'my data bank information to estimate reading demands of jobs 
(Chapter 3) is based upon an actual job reading task— the Primary MOS Enlisted Evalua-. 
tion Test (MOS/ET)— but it also reflects a combination of reading ability and job 
knowledge. In addition, this test is a low-frequency job reading task, and is not 
representative of the day-to-day reading tasks performed by man in the course of doing 
their jobs. ' ^x- 

In this chapter, a third approach for determining reading demands of jobs will be 
described. In this approach, the reading ability requireme,nt of a job. has been conceptu- 
alized as the ability to perform the day-to-day reading tasks of that job^that is, to obtain 
that information required to do the job from the standard printed job reading materials. 
The correlational analysis method was used to assess relationships between the reading 
ability of men in the Army, as established by standardized, grade-school referenced tests, 
and their, performance on Job Reading- Task Tests for three Army MOSs: Vehicle 
Repairman, MOS 63C; Unit and Organization Supply Clerk, MOS 76Y; and Cook, 
MOS 94B. Additionally, this research has developed standardization and normative data 
, on Job Reading Task Tests (JRTT) based on Army recruit input 

A second objective of : this research was to determine the" effects on JRTT perform- 
ance of specific aptitude for / job (defined as Jhaving been assigned to -training for th^t 
job, which presumaibly reflect;^ prior interest, information, ability, and experience in that 
job area), and also the combined effects on JRTT performance of having been assigned to 
a job area, and having completed MOS training in that job area. 

Finally, this research has produced data showing relationships among AFQT, 
Standardized Reading Test (SRT) performance, JRTT, and end-of-c6urse MOS academic 
scores. 



' - ; DEVELOPMENT OF JOB READING TASK TESTS 

CONCEPTUALIZING JOB READING TASKS . 

Job tasks can be roughly categorized into those for which 'reading is an inherent, 
directly involved paxt of the task, sucri as reading incoming correspondence to determine 
appropriate action, and those for which reading is not an inherent* aspect of the task, 
sucji as changing a tire on a truck. In the latter instance, however, written manuals may 
exist telling exactly How the tire is to be changed, and the formal, prescribed job task 
may be to change the tire in accordance with the directidns in the manual. While reading 
skill is^not needed to ^perform the. ultimate task (changing tJ^e tire), there is an enabling 



task-reading the manual— involved in making certain that the tire is changed according to 
the specified procedure. ». 

Most Army jobs appear to contain tasks of this nature; for most tasks and jobs there 
is an appropriate manual or regulation that provides step-by-step directions for per- 
forming the tasks. Although many of these tasks can be learned by *'show-and-tell," and 
hence do not requii;e that the person be- able to read, reading the manual is a part, not 
always explicitly recognized, of the formal job task. Thus, in the Army, to say that a 
particular task requires no reading skill may indicate failure to recognize the fdrmal job 
requirement,.. On the other hand, to always recognize \he formal task requirement would 
be tantamount to asserting that practically a// Army tasks require reading skill and hence 
are job reading tasks. ' 

The foregoing cqmrftents highlight one of the conceptual problems encountered in 
attempting to identify jo6 reading tasks. If the officially prescribed job tasks form the 
basis for identifying reading tasks, then reading task tests may be constructed for tasks 
that, in fact, are not performed on the. job. Thus, supervisors or management people 
are asked to determine what materials a man must^ be able to re.ad and use, they are 
likely to respond in terms of the formal job prescription, or what they'believe, ideally, a 
man should be able to read and comprehend. 

In the present research,^an approach has been used to determine job reading tasks 
that provide a sample of reading- tasks reported by job incumbents interviewed at their 
job sites. This approach ignores formal job prescriptions, and concentrates instead on the 
day-to-day. reading tasks that men perform— whether in accordance with doctrine or not. 
It also focuses directly on reading tasks, rather than on job tasks for which reading might 
be simply an enabling skilL This procedure greatly 'compresses the tinie, cost, and effort 
that would otherwise be involved in job/task analysis. 



IDENTIFYING JOB READING TASKS < 

The identification of job reading tasks and the construction of JRTT were 
performed .under Work Unit REALISTIC. Methodology, test development, procedures, 
and subjects have been described in detail in ' the technical report (1^) and will be only 
summarized here. A ' 

To determine job reading tasks, men'^n the three _MOSs (63C, 76Y, and 94B) were 
administered structured interviews at their job locations. In the interview, each man was 
jasked to give five examples of the times during the "past month or' so" when he had 
been doing some job task and had had to (a) consult some printed job' material, (b) tell 
what information he had 'been seeking, and (c) describe the job task he had been 
performing. Then he was asked to get the manual or other printed job material, to locate 
the exact page or part he had used, and to show the interviewer the specific material 
needed to obtain the desired information. This process was repeated until either five 
instances had been described or the individual could give no more examples. In any 
event, he was not pressed to give more than five examples. 

The men interviewed were first-enlistment men with total time on the job ranging 
from 1 to 18 months and the data refer to job reading tasks for entry and apprentice 
level job performance. 

There were 30 men interviewed in the Supply MOS, 48 in the Cook's MOS, and 85 
in the Repairman's- MOS. The men represented three levels of literacy skill^— grade levels 
4 to 6.9, 7 to 8.9, and 9+— with approximately equal distribution over the three levels. 

^ Determined by prior administration of the Survey of Reading Achievement, Junior High Level; 
California Test Bureau. . . ' 
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CONSTRUCTING JOB READING TASK TESTS / 

As .mentioned previously, each of the men interviewed was asked to ,'cite five 
instances in which he had usedi printed materials in his work in the last monjth or so. 
Copies of the printed materials /cited by the men in/the three MOSs were subsequently 
obtained. A schemre was devisee^ by which each page or section of reading materials cited 
could be classified by the "content type" of information it displayed. The classification 
system used for categorizing/the various materials is presented in Table 11. { 



/ Tableau 
Jiob PrSnted Material/ContentrType Categories 



Category { 



Definition 



1. Tables of Content and Indexes 



2, Standards and Specifications 



3, Identification and Physical 
Description 

4. Procedural Directions 



5. Procedural Check Points 



6, Functional Description 



Cphtent designating the location of information within 
. a publication. ; 

Content setting forth specific rules or tolerances to 
which task procedures or the completed product 
must conform, - | 

Content attempting to symbolically represent an object 
via gn identifying code (stock number, nomenclatureK 
and/or by itemizing its distingui:>ning physical. attt;ibutes. 

Content presenting a step-by-step description of how 
to carry out a specific jotb activity. Essential elements 
are equipment/materials/ingredients to be used, ^rid 
how they are to be used, with presentation organized 
in a sequential step-wise fashion. , l 

Content presenting a key word. or highly surrimarized 
version of what should be done in carrying out a task 
rather than how it should be done. This content differs 
from the content classiiBed under Procedural' Directions 
in thatit assumes the user'^knows ho</v to carry out the 
steps once reminded that^he step exists and/or reminded 
of the decision factors that determine whether the step 
is required. . ' 

Content presenting an ope rating* (cause and effect, dependency 
relationships) description of some existirjg physical system - 
or subsystem, or an existing administrative system or subsyste 



m. 
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In using this classification scheme to construct reading task tests, the printed 
materials cited by the men ^in each MOS were sorted into the six different content 
categories. Setting aside Category 1 (tables of contents and indexes that were obvious and 
simple to . classify), the materials in the remaining five categories were sorted inde- 
pendently by two judges, who agreed on 87, 80, and 96% of their initial judgments in 
the Repairman, Supply, and Cook jobs, respectively. However, in the process 'of sorting, 
materials, difficulties of the classification scheme became apparent. For instance, should 
the unit of classification be based upon a* line (sentence or two) or a paragraph, or a 
major subsection of a technical manual? How should pictorial materials be classified? To 
expedite the present research, materials were classified on the basis of the major 
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suiHirdtioh of a publication. Thus, a section that gave the procedures for filling out a 
frm: was classified Procedural Directions, even though standards and specifications, may 
;.ia.v:^r 3een given in the material. 

Frorn the data about the kind of information a man had been seeking when he used 
the material, and with copies of the printed materials cited, job-related reading task tests 
were constructed. These tests represented the most frequently mentioned reading material 
CL^ntent types, ''and required the man being tested to find the kind of information from 
the materials that job incumbents reported seeking. No prior knowledge that was^pecific 
to the job was required for answering any of the questions. Three separate tests were 
constructed, each using job-specific MOS materials. 

Table 12 lists the subtests in each job reading task test. The variety in the tests for 
the different jobs reflects the variety of different content types tited by men in the jobs. 
Where it was possible to complete the Teadability index, the difficulty leyel is given in 
terms of the modified Flesch -readability formula (4) for both the job material and the 
test material. In all measurable cases, the difficulty level of the materials exceeded that of 
the test questions. , 

Table 12 



Content Types and Difficulty Levels of- 
Job Reading Task Test Materials and Test Questions 



\ \job 

1 \ 


Subtest Content Type^ 


Reading Difficulty Level^ 


Job Material 


Test Question 


RepalrmaV- 


A 


1 


N/A 


8-5. 


(MOs\63C)^ 


B 


2 


N/A 


8.5 


\ ; 


C 


4 


14.5 


8.5 




• D 


.4 


N/>2i 


8.5 




' E 


4 . 


14.5 


11.0 




F 


6 


16- 


N/A 




G 


5 


14:5 


8.5 


Supply Clerk 


A 


1 


N/A 


6.0 


. (MOS 76Y) 


B 


2 


' N/A 


8.5 




C 


. 3 


N/A 






D 


4 


10+ 


11.0 ■ 




E 


5 


8.5 


7.0 


Cook 


A 


1 


N/A 


5.0 


(M0S34B) 


B 


4 


N/A 


7.0 




C^ 


4 


7.0 


6.0, '■ 




D 


A 


8.5 


6.0 



^Content types follow the numbering in Tabic 11. 
'^Readability levels in school grade equivalents. 



The general nature of the readmg task tests is. shown in Figure 4, using a portion of 
the Cook Index test- Questions about the job reading material were presented on the 
right side of the test booklet and the job-related reading materials were presented on the 
left side. A similar layout was used for all subtests. 
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THE RELATIONSHIP OF READING ABILITY TO 
JOB READING TASK TBT PERFORMANCE' 



The relationships of general reading ability to performance on the JRTT were 
evaluated by administering the JRTT for each MOS and a standardized reading test (SRT) 
to three groups: - ' . 

(1) An^unselected sample of several hundred Army recruits at the Fort Ord 
5,eception Station, referred to as the RS group. ^ 

(2) An unselected sample of sever^ hundred men in their first week of Combat 
Support Training (CST) for Vehicle Repairman. Supply Clerk, and Cook, deferred to as 
the Pre-CST group. 

(3) An Unselected group of several tuioidred men in their next-to-last week of 
MOS training, referred ta as the Post-CST group. 

In addition to the administration of the JRTT and SRT, AFQT and end-of-course 
grades were obtained fi^om administrative files where possible.^ With the latter grades, it 
was possible to compute validity coefficients for the JRTT, SRT, and AFQT. 



MEANS, STANDARD DEVIATIONS, AND NUMBERS TESTED 

r For each MOS, Table 13 presents data for AFQT, Standardized Reading Test (SRT) 
performance un reading grade, level (RGL) scores, and scores for each subtest and a total 
score on the Sob Reading Task Tests (JRTT). 

A brief comment should be made to explaia differences in Ns for the three groups 
tested. The differences in the RS groups are small, and represent losses caused by the 
jnabiiity to t^btain the information from records, and failures' to attempt the test. The 
larger differmces bet\sreen RGL and JRTT for Pre- and Post-CST groups who took the 
Cook's JRTT resulted from the fact that, after some of the men had been tested on both 
the SRT and JRTT, a sufficient distribution csf- RGL scores was obtained to permit a 
study of the relationship between the two test performances. However, to obtain job 
information, the Cook's Job Knowledge test^ and the JRTT were administered to 
additional^ nveja. Thus, more men were tested on the JI^TT than on either the SRT or Job 
Knowledge test, resulting in the differences in Ns for the Cooks. A similar explanation 
holds for thi^ Post-CST. Repairman's group: im this c^e, the Job Knowledge test for 
Repairmen^ was not administered to Pre-CSI: men. The remaining differences in Ns for 
any group f >r AFQT, RGL, or JRTT also ais caused by such things as the failure to 
obtain data flropi records. . ^ 

The major data of Table' 13 deal with the differences between mean JRTT perform- 
.ance for tiie RS, Pre-CST, and Post-CST groups. Presumably, the JRTT scores should 
increasg in that order since the RS group is an unselected group, the Pre-CST group was 
selected for their MOS training because of special aptitude for that work, and the 
Post-CST group \yas selected for special MOS aptitude and had completed MOS CST 
training. As the data indicate, although' 'there is some tendency for th6 scores to increase 
in 1 the expected manner, there are many reversals, and those changes in^ the expected 
direction are trivial and may be the result of differential skill levels in general reading/ 
verbal aptitude, as is suggested by an examination of the RGL and AFQT scores. 

A more complete- indication of the influence of selection, and of selection plus 
training on JRTT.; performance is given in Figures 5, 6, and 7, which present mean 

'• • • 

^Developed in HumRRO Work Unit UTILITY, 13. 
^ Developed in Work. Unit UTIUTY. 
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percent correct scores for men of differing reading grade levels in tfe ^IS,. Pre-CST, and 
Post-CST groups. The major findings shown is that, while in general :>fe MOS training 
appears to produce better JRTT performance than that exhibited by ES or Pre-CST 
groups, poorer readers exhibit the most gain. The fact that jkisi: isHiiiEag improves 
performance on the JRTT, at least for the poor readers, attests to riee -^dity of the 
JRTT ^s measures of job-related reading skills. 

A second niajor finding presented in these figures is that genesij^re^ding ability is 
highly related to JRTT performance for all three groups. This lis furlnirr^izi^dicated in the 
data reported in this chapter. 

/' ■ ■ • ' ■ . 

CORRELATION DATA 

Tables 14, 15, and 16 present, for each MOS, intercorrelation znat :ces for AFQT, 
SRT (RGL), JRTT and subtests, end-of-course academic (EOCA) gram, ondj for Repair- 
men and Cooks, Job Knowledge test results. These data are presentediissparately for RS, 
Pre-CST, and Post-CSf groups. . - 

Exarriining the three tables, it is seen that SRT(RGL) is about equally correlated 
with AFTQ and JRTT, with rs ranging from .66 to .82 for SRT and AFQT, and from .65 
to .80 for SRT and JRTT. Thiis^ to a large extent, these three instruments appear to be 
measuring similar skills. This is further evidenc ed b y the somewhat lower, . j^at consistently 
positive correlation coefficients for AFQT and JRTT. The somewhat lowsrrs for AFQT 
and JRTT. than for AFQT and *SRT may reflect the fact that, whereas:;:liae AFQT and 
SRT were constructed to. discriminate among testees, the JRTT was designed to measure' 
ability to perform job reading tasks and was not designed to show differences: among the 
men tested: 



rsj n 



o o 



»n 23 lih o? mo? 

o 2 o CD o o ' 



CO "tf- cn 
in 



XT in 

05 — 03 
— fNJ CO 



— cn CO 
o I- 
c>j pg r>J 



:r oj o 

C3 1— t— 
«N 0« TJ- 



»- O) O 
O — — 
fS CN ^ 



»- tfj O 
O «- O 

CM ^ 



OTO ^030 — C7SO — 
O^T- Of— f— o 



CO O) oo 
in ^ IT) 



m CO 

00 P«» oo 



WcNJoi ^^icin inco 



oj ^ in 

OS »- ,CD 

— rsj CO 



5 25 

f>j rsj oj 



Vf <— ' 

O »— »— 

cvi rsj 



o> o 
o — — 
CN CM ^ 



— C7> O 
O »- ^ 
C>J C>J ^ 



07C3 r- cnf o — oio 



CD ^ in 

CTJ »- CO 

— CM ro 



— cn CO 
O .- 
CVJ^t^i CN 



!^ o 
o »— 
CM fvj <• 



*— OS o 
o — 

CNJ CNJ ^ 



1- C3 O 
O ^ C3 
CN CM ^ 



05 O »— CT) O 
CN ^ CN CN ^ 



m ^ in 

— U> 
— CM CO 



^ OJ C» 

o »- 'd' 

fN CM CN 



O »— ^ 

CM CN ^ 



— 03 O 
O «™ — 

CN CN 



— CT> O 
O *- 
CN CM ^ 



O tj- 

U) t>l CO 



05 ^ in' 
OS — to 

— CN CO 



— O) CO 

o f- 'i- 

CN CN 



5 2 2 

«N CN <■ 



— O) O 
O — — 

-CN CM it 



— <n o 
o ^ 

C*< CN ^ 



cn ^ in 
OS — to 

— CM CO 



— o) oo 

O f- it 
CN CM CM 



'I OS O 

o ^ ^ 

CVI CN <■ 



— C7> O 
o — — 

CN CN ^ 



cn ^ in 

OS f- CO 
— CN CO 



5 23 

CN CN C^i 



(o 00 in 
CO m CO 



OJ ^ Ui 
OS CO 
— CM CO 



5 23 

v'N CM CN 



C7S ^ cn 
cn *- *- 

— CM CN 



5 H te 

C/> CO o 

cc £ 



CM cc a 



i h; fe 
O CO w o 



2* H te 

CO CO C5 



CO CO 



cc 1^ 



■ < 



I 



fi h 

« <^ S 
K a: £ 



Q 



2 H te 

C/5 CO o 

oc £ a. 



u 

oc £ 



5 I- 
c/> «3 o 



< 
o 
o 



|2 



< 

4-* 



o 
H 

t 





O 








LO 




CNI 


CNI 




CO 


0). 










CO 






CO 


LT) 




CO 

* 


in 




CO 


CO 




'^t 






CO 






CO 


Csl 




in 






p^ 






C3i 


00 




O) 


00 




O) 


00 




Oi 


00 






(D 




CO 






CO 


O 




CO 


o 




CO 


o 




CO 


O 




CNI 


CNI 




CNI 

• 


CO 




CNI 


CO 




CN 


CO 




CN 


CO 




CN 


CO 






CO 




CO 


in 


00 


P^ 


00 






CN 


00 


^ 


in 


CN 


P^ 


CN 






• 


in 


in 
• 


in 


p^^ 


P^ 

• 


p^^ 


in 


in 


^. 


CO 


CO 

• 


CO 


in 


CO 


CN 
• 






CO 


CNI 


o 




CNI 


CNI 


in 


CNI 


CN 


ID 


CN 


CN 


in 


CN 


CN 




CM 


00 




CNI 






CNI 






CNI 






CN 






CN 






CM 


CNI 


CNI 


CNI 


CO 


CO 


CNI 


?5 


CO 


CNI 


CO 


CO 


CN 


CO 


xn 


CN 


CO 


CO 


O 


CNI 


CJ) 


p^ 


CNI 


in 


in 


CO 


O) 


in 




p^ 




o 




CN 


CM 


CO 


(O 


in 




CO 


CO 


in 


00 


00 


p^ 


in 




CO 


in 




5r 


in 




CN 


T— 


in 


CO 


CNI 


O 




CNI 


CNI 


in 


CNI 


CN 


U) 


CN 


CN 


in 


CN 


CM 


in 


CNI 


00 




CNI 






CNI 


r— 




CNI 






CN 


r- 




CN 






CM 


CNI 


CNI 


CNI 


CO 


CO 


CNI 


CO 


CO 


CNI 


CO 


CO 


CN 


CO 




CN 


CO 


CO 




CM 


in 


CNI 


tn 


o 


P^ 


CNI 


00 


00 




in 


00 


P^ 












C9 




CO 




CNI 


P^ 


CO 




in 




CO 




CO 


CN 










in 


CO 


CNI 


o 




CNI 


CNI 


in 


CN 


CN 


LO 


CN 


CN 


in 










00 




C>J 




\ — 


CNI 






CNI 






CN 




I — 


1 






CNI 


CNI 


CNI 


CNI 


CO 


CO 


CNI 


CO 


CO 


CN 


CO 


CO 


CM 


CO 


CO 








V 






































CJ) 


CNI 


CO 


p^ 


CNI 


CO 




O) 


CN 


CO 


p^ 


*■ 












• 


CO 

• 


• 


in 




in 


CO 

• . ■ ' 


in 


in 


in 


^. 


CO 

* 
















in 


CO 


CNI 


o 




CNI 


CNI 


in 


CN 


CN 


LO 














CNI 


00 




CNI 






CNI 






CN 


















CNI 


CNI 


CNI 


CM 


CO 


CO 


CNJ 


CO 


CO 


CN 


CO 


CO 














CO 


in 


O 


CO 


CO 




r~ 


CO 






















in 






CO 


in 


in 


00 


P^ 


p^ 






















in 


CO 


CNI 


O 




CNI 


CN 


in 




















CNI 


00 


p^ 


CN 






CNI 
























CNI 


CNI 


CNI 


CNI 






CNI 


CO 


CO 




















UO 


6 


CO 


00 




o 


























(O 


CO 


lO 


P^ 


p^ 


p^ 




























in 


CO 


CNI 


o 




























CNI 


00 


P^ 


CNI 






























CNI 


CNI 


CNI 


CNI 




tn 



























CO o 

in in 



CJJnOO 

CO o 
CN CO 



in CN 
in ^ 



CN CN in 
CN 

CN CO CO 



CN 



CJ) CO 
CO o 
CN CO 



o 

CM C 

H 

OC 
CO 



O) in in 
ps. ps. 



r- CO CO 

CN 00 

CN CN CN 



a 
< 



o 

5 



(0 
6 

CC 



o 



(0 

C/3 
d 
OC 



£ 8 

Q. 0. 



^ W 

g £ S 

OC Q. OL 



lis 

CC CL 



CO 

CO 



C/3 
O 



o J% </> 

« £ o 

CW Q- 



OC Q. 0. 



0 

.2 



a 



CM c 

C/D 



CO 



< • 


OQ 


o 


o 


LU 




in to 

^; 


/ CD t; 


to . 


CD H 

|2 



C/) 

O 
S 

O 

o 
O 

s ® 

K CO 

c 
o 

JO 
Q) 
k. 

^ O 

o 

k. 



9 

Job Know. 




z 


< ' 
0 

LU 




z 


a 

■ <u 




z 


U 

0) 




z 


CD 
u> Ik 

0) 




z 


< 




z 


! 3 
JRTT Tot. ' 




z 


. -J 
O 
w £ 

1- 

oc 


\^ 


z 


u. 




z 


> 

■ - a 


Variable 



to 



o (o 

CO <D 



^ CO 

10 in 



(N O 



00 CD 
•d- to CO 



•d- c>J N 

C?) O) 
CM C^4 



o 

CO 



CD T- 

in in 



^ CN CM 
C3) CJ) 
*- CM CM 



00 Ifl 

tn CO CO 



CM CM 
O) O) 
T- CM CM 



in 00 
^ tn ^ 







CM 












CO 






CO 


C7) 






CO 








cq 






in 


cq 




'^r 


in 




in 


cq 




in 


in 








in 






in 






in 






in 


i~ 


in 




CD 








CO 


00 




CO 


00 




CO 


ot> 




CO 


00 


CO 


00 


CM 




















T— 










.. 1— 








o 




m 


CO 






in 




00 






00 


o 


00 


CO 










in 


in 










CO 








in 


^. 


CO 






CO 




00 


in 




'^OO 


in 




CX) 


in 




00 


in 


C30 


in 










O) 


O) 




O) 






0) 


O) 




O) 




O) 


CJ> ■ 






CM 




CM 


CM 




CM 


CM 




CM 


CM 




CM 


CM 










00 


LP 




.00 








o 










fTi 
U) 


















iq 


CO 












in 


^. 












in 


CM 


00 


in 


CM 


00 


in 


CM 




in 


CM 


C30 












0) 


CM 


O) 


CD 


CM 


O) 


O) 


CM 






CM 


O) 










CM 




CO 


rvj 




CO 


CM 




CO 


CM 






CM 












o 




cn 


00 


O 


00 




in 


C7) 
















CO 




C7) 


oq 


cq 




cq 




cq 


in 


















in 


CM 


00 


in 


CM 


00 


in 


CM 


00 


















C7) 


CM 


o> 


C7> 


CM 


C7) 




CM 


CD 
















CM 




CO 


CM 




CQ 


CM 




CO 


CM 
















CO ' 


CO 


in 


o 


oo'rs 
























in 








in 


in 


in 
























in 


CM 


00 


LQ 


CM 


00 
























C7) 


CM 


C7) 


at 


CM 


cn 






















CM 




n 


CM 




CO 


CM 























00 CO ^ 

CO CO- 



CO CO 

00 00' CO 



^CMCM cor%r^ incMoo 

0)0)»— C35 00^. 0)CMO) 
CM CM CM »- CO CM 



^ CO ^ 
CO CO IX? 



00 O CO 



^ CM CM CO fN. 
0) O) «— G) CO ^ 
CM CM t- T- CM 



O CM CO 

00 CO 



CM CM CD 
C3) CO 2^ 



rtJ H to 
O 



u 

0) 

DC 



2. S 

0. 0. 



^ £ 1 

0. 0. 



a: 



£ o 

0. 0. 



a 

< 



CM CC 
C/) 



0. CL 



0. 

CC CL. a. 



CC CL. CL 



8 • « 
oc a. a. 



4§i 

Si S 

QC CL 0- 





< 


CO 


o 


Q 




CO t ^ 

CC h- 
-o 




in ^ 

1= 




|2 


00 



< 
o 
o 

111 



4 



IIMTERCORRELATIONS AMONG JRTT SUBTESTS AND 
REMAINING VARIABLES 



Generally speaking, the intercorrelations among the subtests for each JRTT are 
moderate and positive. The lowest rs are obtained with the Post-CST data, which 
probably reflects the most homogeneous nature of the subjects (Table 13, Columns 5, 
6, 7) over those of the RS -^and Pre-CST groups, and the near-ceiling attainment levels 
- (Table 13, Columns 2, 3, ,4) .of many Post-CST men on subtests with small point values. 
For the -Reception Station (RS) group. Table 17 presents, the relationships of each 
subtest with the sum of all other subtests in a given MOS JRTT. The rs are* all 
moderately high, indicating that each subtest is measuring the ,same capacities as 
measured by the sum . of the other subtests. These correlations are quite high considering 
that the JRTTs were not designed tointr^ase the ^variance among subjects, which would' 
tend to enhance these rs. These data, and those of the preceding paragraph, suggest that 
each subtest provides a moderately effective, measure of general reading ability, as well as 
a measure of job-related reading skills. ' . . 

'1 , . ' ■ 

'; " ^ Table 17 ' • ' ; 

Correlations of Each Job Reading Task Test Subtest 
With the Sum of the Other Subtests 
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RELIABILITY OF JRTT ^ . , C- 

The testing schedule for the Pre-CST and Post-CST groups was such that a small., 
sample of men in each^MOS school were included in both groups. For the Repairman, 
Supply Clerk, and Cook schools, the nunibers of men for whom both Pipe- and Post-CST 
scores, were available yirere, respectively, 36, 98, and. 37, and test-retest reliabilities for the 
JRTT were/respectively, .85, .74, 'and .80. These indicated acceptable levels of, stability 
for the* scores on the test instruments. ' " " 

oVALIDITY OF AFQT , SRT , AND JRtV FOR 

PREDICTING END-OF-COURSE ACADEMIC GRADES , . , ' 

As mentioned before, end-of-course academic (EOCA) grades were obtained for t'he 
Pre- and Post-CST groups, and intercorrelations for these grades and AFQT, SRT, and 
JRTT are presented in Tables 14, 15., and 16. Ih these tables, it should be noted that the 
coefficients for the AFQT and Pre-CST groups with the • EOCA are predictive validity , 



coefficients, because the tests were administered seven weeks prior to thre awarding of an 
EOCA grade. On the other hand, the coefficients for the SI^T and JRTT with EOCA 
■grades tor the Post-CST groups^e concurrent validity coefficients, because the reading 
. tests were admimstered during the last* week of CST training, when final EOCA gr.ades 
were assigned. •■ , ' 

Overall, it is clear that the three predictor tests show moderately strong, positive 
correlations mth the EOCA. As expected, the coefficients for the various JRTT subtests 
are less than for the. JRTT total scores, primarily reflecting the reduction in number of 
itenis and lower reliabilities of the-subtests. 

Of note is the fact th^t the AFQT and SRT, both non-MOS-related measures of 
reading, are as effective as. the JRTT in predicting-aeademic achievement in MOS training, 
even though the JRTT reflects job-specific reading content! and format. Thus, whila the 
■ JRTTs have greater **face" validity than do the AFQT and SRT, the latter instruments 
i permit the same efficiency of prediction of MOS CST achievement as do the JRTTs. 
\ One remaining piece of evidence concerning the validity of the AFQT and JRTT 
J comes from the Cook's and Repairman's MOSs, in which a number of men were 
administered the Job Knowledge tests" developed' by HumRRO Work Unit UTILITY: For 
the Cook's MOS, the Job Knowledge test and JRTT were administered to men in both 
the Pre- and Post-CST groups. For the Repairmen, only the Post-CST group took the Job 
Knowledge tests. Table 16, Column^ 9, presents intercorrelations for the AFQT, JRTT, 
and Job Knowledge tests for Pre- and Post-GST in the Cook's MOS. Table 14, Column 
12, presents comparable data for the Post-CST Repairman. In all cases, the rs are positive 
and moderately hi^h, suggesting a large component of reading or other verbal ability in 
performing on the Job Knowledge measures. This is best evidenced by the Cook's 
Pre-CST data, in which the men had not been taught any job |cnov/ledge (Table 16), yet 
the .^correlations of AFQT and JRTT with the Job Knowledge test performance are .53 
anu .62, respectively. ' ^ 

USE-OF JRTT TO DETERMINE JOB LITERACY DEMANDS ^ 

Figure 8 demonstrates how the JRTT and^^SRT relationships might be analyzed to 
defermine the reading skills required to perform the job reading tasks. It shows the 
percentage of men in the Post-CST groups who scored at or above 70% correct on their 
MOS JRTT, in relation to the SRT jading ability level of the men. The horizontal 
dotted line crosses each curve at a point where 80% of the nien obtain the criterion score 
of 70% correction the, JRTT. The vertical dotted lines mark the reading grade level on 
the X-axis at which 80% of the men score 70% or more correct on the JRTT. Thus, if a 
rule were adopted that stated that the reading level required ior satisfactory performance ' 
' on JRTT is the leyel at which 80% of the men score 70% or better on the JRTT, then 
the reading requirement for both the Cook and Repairmen's MOSs would fall in the 
range of 7 to 7.9, and the Supply MOS would be near 10. 

•Obviously, the estimates of reading requirements will change as the decision rule is 
changed. At the limitin'g case wherein 100% of men are expected to achieve 100% corrisct 
on the JRTT, it is clear from Figure 8 that a 10th grade level would be required for 
Cooks and Repairmen, while' the Supply Clerk's reading, requirements would be estimated^ 
well above the 12th grade level. Decisions cohcerning how low criterion levels might be':^^' 
setr must be based upon additional knowledge, "suchv, as the supply and demand charac- ? 
' teristics of the nianpower situation; whether or not literacy training will be provided, iand, 
what additional information is available concerning the reading demands of jobs (Chapters ' 
2 and 3 and in. the REALISTIC^ 1, research). Certain of these considerations are 



Percent of Post-iCST Men at Each Reading Grade Level Scoring 70% or 
Higher on the Job Reading Task Tests for Three MOSs 
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Chapter 5 
DISCUSSION 

This chapter presents a brief review of many of the manpower considerations and 
problems that led to the initiation of Work Units REALISTIC and READNEED. Certain 
conceptual and procedured problems in specifying reading requirements of jobs will be 
discussed, and four general approaches mvolving seven specific methods that have been 
used for problem assessment vnil be examined. 

STRATEGIL^ FOR COPSNG WITH MARGINALLY LITERATE MANPOWER ^ 

Several strategies for coping with the problems of illiteracy (or low literacy) have 
been used by the Army, and other military services, at various times. 

Nonacceptance of Illiterates. Qne way to overcome the problems resulting from low 
literacy skills in inductees is to avoid them. From time to time, the Army has raised its 
admittance standards with the result that large numbers of marginally literate men were 
excluded from service. In general, standard^ have been raised during intervals of relative 
military quiescence and lowered during periods of military activity, such as the Korean 
; and Vietnam conflicts. • 

There are, however, several problems associated with the strategy of non- 
acceptance that limit its fruitfulness. For one thing, as vdth many other abilities, it is not 
a simple matter to acciurately assess a man's literacy skills at the selection station. Hence, 
large numbers of potentially useful men may be turned away, while some who are not 
useful may be accepted. \^ 

The probjem of accurately selecting men on the basis of their literacy skill is 
compounded by the fact that,, until thevpresent research, no attempt had been made to 
accurately identify literacy skill levels Required by Army jobs and training schools. 
Therefore, there were no adequate criteria tqi selecting cutoff points on selection tests. 

Both of the foregoing problems^are concerned with assessment— the first v^th 
assessing the man, and the second with assessing the job requirehients. A third problem 
affecting the usefulness and. desirability of the siprategy^ of 'noniacceptance concerns 
tridning. Manpower needs are such that it may^become necessary, under conditions of 
large-scale mobilization, to enlist marginally literate.^ men. If these men are not accepted 
^ during peacetime and the training" methods needed to -keep pace v^th technological 

change developed to ^effectively train such men, then' new training techniques and 
methods will have to be developed imder the stress of mobilization, when expediency and 
not-effectiveness, may be the predominant training motiv^ 

Perhaps the most signilBcant results of the nonacceptance into the Army of 
men -who are marginally literate is that a large and needy se^gment of the population is 
not able>to reap the benefits of tho training, education, social xievelopment, and practical 
experience that accompany Army service. This problem was unoerscored in the sixties by 
the initiation of two Department of Defense projects. In Project 100,000, announced in 
1966, -all the services accepted individuals scoring in the lower half of Mental Category IV - 
on 4he Armed Forces Qualifications Test (AFQT).^ In Project TRANSITION, initiated in 

^ * Men in Mental Category IV have AFQT scores of 10-30. ' ' 
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1967, the military services provided in-service training and education to low-aptitude (and 
other) servicemen in order to prepare them for return to civilian life. 

Remedial Training. A second strategy sometimes used by the Army, and the other 
services, in an attempt to solve the problems resulting from low literacy is to provide 
remedial training in reading. These training attempts have been .thoroughly critiqued in 
the Department of Defense review of marginal men (14). That review indicates that 
literacy training, for the most part, has been ineffective in improving the on-the-job 
performance of low-aptitude men. It should be mentioned, however, that those literacy 
training progranis did not have recourse to the improved technologies in instruction 
curreritly available. Remedial training now might be made more effective m improving 
" literacy skills enough to reflect stlch improvements in job performance. 

A further limitation to the remedial training programs mentioned is that they 
were not directed towaxd using literacy skills in a given job, but were geared to the 
attainment of general fourth-grade levels of reading skill and limited military vocabulary. 
Remedial training programs- ciurently in progress are directed at the attainment of 
fifth-grade levels of literacy skills. However, since the^e has been inadequate knowledge of 
literacy skill levels necessary to satisfactorily perform given Army jobs* (inc^ding training 
courses), remedial training objectives could not be and, as the present research indicates, 
were riot stated accurately. 

, Limited Assignments. The problem of using marginal literates also has been decJt 
with by assigning these men to. MOSs that have "minimal" requirements for reading. This 
strategy has not worked well (14) for two reasons. First, as with the other strategies 
reviewed, adequate definitions of the requirements for reading skills in different jobs have 
not been available; hence, it has not been possible to accurately state '^minimal" 
. requirements for reading skills. 

A second- difficulty is cpncerned with selecting job proficiency * levels for 
establishing reading requirements. {^Individuals qualified for entry-level jobs* are not 
necessarily qualified for advanced level jobs, yet in most instances advanced-level job 
positions are filled with personnel from the entry-level Jobs. The Si.ssignmen(t of a man 
with the marginal requirements needed for an entry-level job may result, perhaps because 
of combat casualties, in his promotion tp a leadership position, with possible devastating 
effects for -him aiid the men he Lads. 

Another difficulty with tl;ie policy of assigning the" marginally literate to an 
MOS having relatively lew requirements 'for literacy and arithmetic skills concerns the 
overall effectiveness of an MOS. This may b$ reduced if the MOS becomes flooded with 
marginals. Therefore^ some means '-^are necessary for distributing these men equitably 
among the suitable MOSs and the jobs within MOSs. I* • 

Modification of Training and Job Requirements. A fourth strategy for dealing with 
the problems of low literacy skills is to redesign training and job materials to minimize 
the importance of such' skills. Under £his strategy,, training schedules and practices may be 
modified to" meet the skill levels of different individuals. Thus, vmtten instructional 
material may prove, adequate for certain individuals, but the same -instructions might best 
be presented in some other way for iiidividuals having relatively low reading-skill levels. 
HumRRO Work Units APSTRAT, SPECTRUM, and EVATP have-^^tudied problems of 
individualized training techniques, and the development of training techniques especially 
effective VTith lower-aptitude men. 

.Although training courses can be modified to reduce the need for literacy skills, 
it should be noted that if a job requires a certain level of reading skilly then training that 
deemphasizes such reading is unrealistic and inadequate. Since the literacy skill levels 
necessary for satisfactory job perfoi^nsuice in different MOSs have not been kno^yn, the 
minimal levels of reading difficulty in wrritten materials that permit the satisfactory.- 
'accomplishment of a job have been undefinable. . 



As ,\vith the other strategies, literacy skill levels of men must be identified so 
that these men can be assigned to training courses and be provided with training 
techniques appropriate to their needs. There is also the problem of specifying the skill 
levels required by the training and job materials in order to establish the necessity for 
redesign, and to establish the sldll levels required by the redesigned materials. 

ASSESSING JOB READING REQUIREMEIMTS 

A problem common to the four strategies reviewed is that of defining skill-level 
requirements for jobs. For knowledgeable implementation of any one of these courses of 
action, information about the literacy skill levels necessary for successful job performance 
is necessary. Obtaining such information formed the primary, basis for the initiation of 
Work Units REALISTIC and READNEED. In accomplishing these Work Units, four 
general approaches, for assessing literacy demands of jobs have been considered. In the 
following paragraphs, each of these is disjuss'ed briefly. 



ESTIMATE BASED UPON JUDGMENTS OF . 
GENERAL EDUCATIONAL REQUmEltelTS 

One approach for estimating reading demands of jobs is that used by the U.S. 
Department of Labor. Job analysts estimate the level of General Educational Develop- 
ment (GED) required for various jobs, based upon interviews with Job incumbents, 
supervisors, and observation of the. job being performed. A job may be categorized into 
any one of six GED levels, developed, to roughly parallel school-based educational 
development (e.g., a GED of , level 1, is said to approximate the education obtained in 
grades 1-3, and level 2 parallels 4-6th grade education, 15), and a job assigned a GED 
level can be '.said to Yvaye been assigned a reading giade level. This approach to assessment 
of reading requirements of jobs is a judgmental approach that calls for an estimate"^by the 
job analyst.* Reading grade levels assigned by this method to eight Army MOSs studied in 
REALISTIC and READNEED are given in Column 5 of Table 18. 



ESTIMATES GIVEN IN SUMMARY TASK STATEMENTS 

A second approach to establishing literacy demands of jobs, and the approach 
' - s. currently used by military job specialists, is the summary task statement. In this 

approach, supervisors or job analysts prepare a summary statement of the tasks to be 
performed, m the form: "requires ability to- read technical manuals," or "must have 
y i bal ability to read simple forms and records." Such statements give no indication of a 
Specified level of general , reading skill required by the job. Examples of these statements 
from Army Regulation'611-201^ are given in Column 2 of Table 18. 

. Both of the foregoing methods rely upon a job analyst or other responsible person 
to make a summary estimate of a job's reading demands without the use of a carefully , 
articulated statement of the rules to be followed in arriving at this estimate, and without ' 
objective validating observations. For this reason, one may question the accuracy and, 
, appropriateness of such estimates. f . c r ^ 

• i.' ■ •■ ■ ' 

ft ■ ■ • 

^Enlisted MiUtary Occupational Specialties, AR 611-201, 5 Jan 67. 
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READABILITY ESTIMATES OF JOB READING DEMANDS 

Th^ development of readability formulas (Chapter 2) has made possible a third 
method for the assessment of the reading demands of jobs. By applying a readability 
formula to samples of job reading materials, an average reading grade level for the 
materials can be computed, and used to represent the reading requirements of the job. 
This is an objective, mechanical procedure, although it may not always be possible to 
obtain a rejDresentative sample of job materials, or even to determine the proper domain 
of materials from which sampling should be done. In this regard, a major difficulty can 
arise because of the distinction between the formal job task specifications and the actual, 
or informal job tasks that are performed on a day-to-day basis. If supervisors, manage- 
ment, or content experts are consulted to find out what reading materials a man must 
use in doing h;;> job, they are likely to base their statements upon their conception of the 
formal, or even idealized job, and to prepare a list of materials ^which exceed reasonable 
expectations and which are, in fact, notarised in the worli-day activities. This situiUii* 
was encountered in REALISTIC, whr-re, in the Cc jk's AIOS, materials reported by job 
incumbents :is having been read in their work differed considerably from the mau.: is 
identified by supervisors as being thr ^nh reading materials. 

Readinn requirements determined by readability ^analyses are presented fbr 10 MOSs 
in Column 6 of Table 18, Materials for the Cook's IvdOS were identified by interviews 
with job incumbents. The remaining readability estimates are based upon formally 
prescribed materials to be read for MOS proficiency qualification. The' reading grade 
levels (RGL) given are rounded medians. In general, readability estimates appear to set 
reading^ levels higher than the other methods do. This probably occurs, at least in part, 
because • the formally prescribed reading mateirials are very difficult and demand high 
levels of reading skills for effective use. The Cook's MOS, on the other hand, represents 
what '.men use on the job. This material has a lower reading difficulty level than macterials 
sampled that were suggested by supervisors (grade 9.0 vs. grade 11.0). 

ESTIMATES BASED ON TESTING OF PERSONNEL 

A fourth general method foi estimating job reading requirements, and the method 
constituting the bidk of the REALISTIC and READNEED research, is the traditional 
psychometric model for validating selection and classification tests. In this procedure^ 
performance on a reading predictor test is related via correlational techniques to perform- 
ance on a job proficiency criterion test. If the relationship is high enough, cut-off scores 
on the predictor variable can be selected to maximize the probability of obtaining 
personnel who will reach an acceptable)level of proficiency. 

There are two formally recognized procedures for 'making this analysis. One proce- 
dure is to measure the literacy skills bf potential employees, then to employ a,U these 
persons and wait and find out v/ho the successful performers are. The relationship 
between the original measures , of literacy and the probability of being a successful 
performer is then determined, and a required literacy level can be established at the level 
that predicted successful job performance. This is the "predictive validity** psychometric 
model. While this is a powerful approach to the problem, it has the major drawback of 
requiring a considerable time between th? administration bf the literacy tests and the 
subsequent assessment of job proficiency. It is also frequently difficult to convince 
employers to hire all the job applicants so that subsequent failures can be detected. 

A soms-'what^less satisfactory, but more expedient procedure is to use the ^^con- 
current validity'* psychometric model. Under this approach, job incumbents are adminis- 
tered both literacy tests, and job proficiency tests in close temporal proximity. Scores on 
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the literacy test are then related to scores on the job proficiency measures as in the 
•^predictive validity" model. The major drawback to this approach is that the job 
incumbents are, by virtue of being on the job, **successfu]''; ;he poorest job enrollees will 
have departed, and one is wuu a higher level sample. If literacy is, in fact, related to 
job proficiency, then it is likely that men of very low levels of literacy will have been 
removed from the job ranks. For this reason, the relationships determined by the 
concurrent validity model may be attenuated. However, because of the convenience of 
this approach, it was followed for most of thfe REALISTIC and READNEED .research. 

In applying either of the foregoing psychometric models, major decisions are 
encountered in selecting the measures of literacy and job proficiency. In the REALISTIC 
research, we had first to define what was meant by literacy. While \w is cle^x tha; this 
refers to the ability to read, this abi'ivty an ridmLKiure of s- Jls i: 1 in decoding 

written symbol? to s.t;; -* internal '.'ei: pic-, -ntation .iwC. skills and knowledges involved 
i lan^ucige coiuprehensi- n and .ise. Thus, one might be ''illiterate" in-the sense of being 
i .ible Lo read (decode), but be "literate" in the sense of knowing the language base 
upon which the written word is built. On the other hand, one can be "illiterate" in both 
senses— lacking decoding skills and language, sldlls and knowledges. In REALISTIC, in an 
attempt to accommodate the first type of "illiteracyi" literacy was assessed using a 
nonreading, listening test. .In the READNEED research, literacy included only reading 
performance, with no attempt at separate assessment of nonreading language capabilities. 

A second category of problems involved measuring job proficiency. In stating general 
literacy demands of jobs, it would seem desirable to measure job proficiency in terms of 
the tasks in the job that involve reading. One could then construct tests involving these 
tasks, and the performance on them can be related to the performance on the general 
reading tests. However, as mentioned earlier,^ many important job tasks make no 
immediate demands upon reading skills; rather, they have an indirect need for such skills. 
For instance, preparing scrambled eggs makes no. immediate demands for literacy skills, 
but knowing when and how to prepare scrambled eggs does— the cook must read the 
bulletin board to know that eggs should be prepared that day, he must read the master 
menu, and so forth. Therefore, job tasks can be defined in terms of the immediacy and 
directness of the demands for reading skills. Some tasks (e.g., filling out supply forms) 
are inherently reading tasks; others (e.g., setting the spark plug gap in a jeep) may have 
an indirect demand for reading-Hhe mechanic may use a technical manual during his 
school training to learn how to adjust spark plugs, and then never refer to the manual 
again. Other tasks may be learned completely by "show-and-1:eir'--making no demands 
upon reading, but involving learning by listening. 

Because job tasks may make more or less immediate and direct demands upon 
reading, the REALISTIC and READNEED research has examined relationships of reading 
ability to four differfel cypes of job proficiency involving tasks that vary in their demand 
for reading. In both REALISTIC and READNEED, relationships between reading ability 
and performance on Job Reading Task Tests (JRTT) wer^ examined (READNEED, 
Chapter 4). In READNEED, relationships were examined between reading ability and 
proficiency on^. the Primary Military Occupational Specialty/Enlisted Evaluation Test 
(MOS/ET, Chapters). In REALISTIC, reading ability was related to performance of 
experienced job incumbents on two indices of job proficiency: job knowledge, paper- 
and-pencil tests, and job sample performance tests; in which men performed four to five 
hours on- simulated job tasks."^ 



^In "Development of Job Reading Task Tests," Chapter 4. 

^In both REALISTIC and READNEED, relationships between reading ability and supervisor rat- 
ings were examined. In neither case was the relationship sizable enough to warrant further consideration. 



Thus, in the combined READNEED and REALISTIC research, xhere are .four 
criterion measures of Army job proficiency. The JRTT reflects proficiency i"; reading the 
materials reported by pb incumbents in three MOSs to be those that -"-i^;^' v ally used 
in doing their job. They are thus direct r jeasur es of ability to read anc iUm ,r:im y printed 
materials, ^ho JRTT a^. much like a standarr^ized reading test, except tL::a;. ihe contain 
Army material a^id they ask the question? that the job incumbt ' its repoiled n^tzing. The 
job knowlecf^ and MOS r^;! tests al^-o nici, . :'n immediate dernp ^id up^vT. remcr^., ability, 
but, ir add': i>r7. ■hiy ^ :-..re job Lujc vvledge for their satisfactory completion. The job 
sample tescs make little ur no direct demands upon reading ability (excluding the Supply 
Clerk's MOS), except when men chose to use the availa^Dle TMs. 

SELECTING A METHOD FOR 
ESTIMATING ARIVJV Wi^S READING DEIW;ANDS 

\^uch ..f th« L~cv, :\ rr-;:hithods lisred in Table 18 should be used to doi - ' V. ir .t reading 
-.requirements of ^4rmy MOSs? So far as we know, there is no means other than 
rationalized human judgment for selecting one method over the other. An answer 
depends, in part, upon the purpose to be served, and the cost in timo, money, and 
personnel, that one is willing to pay. While there is no unique, finiu^ fj^olution to the 
problem of selecting^ the best method for de^>ermining ^eadingr demand , sii^u-iain features 
of the various methods c:an be made ex^^llci't.^ lo ensure that t\ur' will be ^^^n^^VHared when 
one or the oth.^r rviO^ ^cV: jontempxated. ^ . 



SUMMARY TASK STATEMENTS 

While the method of summary task statements (Table 18, Column 2) is low cost, it 
appears inadequate for any but the most gross screening of input. Without knowing what 
materials are to be read, the level of reading ability needed to re^M and understand them 
is indeterminate and unmeasurable. 



The DOT RGL estimate (Table 18, Column 5) is also relativei\^ Low in cost^ but it 
appears insufficient on several counts for the purpose of determining with usf-ful pre- 
cision Army job reading requirements: ^ 

(1) For Army combat MOSs, there are no equiv^enx civilian jobs and the 
general military-civilian correspondence of jobs must be judgmentaiir estimated. 

(2) Literacy requirements in the applicable raiige are spedned in ^categories so 
broad as to permit only the coarsest differentiation. 

(3) The literacy requirements estimated by this method (such as RCX, 9-12 for 
jobs needing GED level 4) refer to the average or typical overall curricula xm c ontent 
taught in those school years, rather than to a measured level of reading ability. 

(4) There is no statement of the contemplated level of job proficiency for 
either military or civilian jobs. 

(5) Since the current source of DOT codes for Army MOSs provides several 
alternative DOT codes referring to civilian jobs to which the Army MOS is rou^ly. 
analogous, selection among those codes is done on a judgmental basis and, once accom- 
plished, encounters all the problems listed above. ^* 
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FIVE EMPIRICAL METHODS ' 

In sharp distinction to the DOT-RGL procedure, the five empirical methods 
I (Table 18, Columns 6 to 10) of REALISTIC and READNEED yield a full statement of 
the relationship between all levels of literacy skills sjid all levels of some measure of job 
proficiency. The nature of the relationship is empirical- and the data from which it is 
derived are described and subject to verification and extension to other MOSs. Given data 
on the relationship between measures of reading and job proficiency, the specification of 
the reading requirement for an MOS becomes directly dependent upon the level of job 
proficiency specified: the higher the job proficiency standard, the higher the reading 
requirement and idee versa. The judgment as to .what level of job proficiency is good 
enough rests squarely vrith Army management. .Without this jud^ental decision about 
the target level of job proficiency, the reading level required for that job is indeterminate 

Each of the five empirical methods (Table 18, Columns 6, 7, 8, 9, and 10) for 
.determining job reading requirements uses a different criterion measure of job profi- 
ciency. At present, rationalized human judgment appears to be the only method available, 
for selecting one criterion over the other as the measure of job proficiency. The 
judgment, however, is crucial, because, as indicated in Table 18; reading requirements 
may change depending upon the criterion definition pf what level of which performance 
dimension constitutes satisfactory job proficiency. 

Foi want of an authoritative specification of reading matei^ial ^at had to be 
understood for criterion-level job performance, the set of job - information source 
materials listed in the DA PAM 12- Series for each MOS was adopted in^the READNEED 
research. Systematic samples of reading passages from these sets clustered heavily at the 
nth and 12th grade reading difficulty level for each of the seven MOSs studied. This 
reflected the preponderance of materials specified for the formal definition of the job, 
but these v ere too difficult to be used by most job incumbents. Our working decision to 
set the M03 reading requirement at the reading level for the easiest-to-read Half of the 
passages was an arbitrary criterion standard. Considered abstractly, a job reading require- 
ment at this level does not seem too demanding— but that is so only for reading materials 
that • are necessary for realistically attainable, satisfactory] job proficiency, not for an 
idealized^ total job mastery criterion. Certainly the DA PAM 12- Series seems an overiy 
inclusive'and demanding specification, of printed materials that must be read and compre- 
hended in order to perform these jobs satisfactorily. 



Given an autH^oritative, competent, responsible criterion specification of just those 
job reading materialsHhat mus^ be readable with comprehension in order to perform the 
job in a reasonably satisfactory manner, the readfa6///7y 'method would seem to be an 
appropriate one for determining' MOS reading requirements. Setting the minimum 
criterion list of job reading materials is a difficult task; however, implementation needs 
' are minii^al, requiring only clerical time to count the one-syllable words in passages 
sampled from the criterion materials list. Given a listing of what must" be read, the 
readability procedure will easily specify the level of reading ability needed. 

Because of the difficulty in getting agreement on a minimal set of essential job 
reading materials, the*FORCAST readability formula could be put to interim use in 
determining the readability of the separate and more limited reading materials used in 
MOS entry-level training courses. While these do not constitute the total body of jqb 
reading materials, they do constitute a common core of the most basic reading materials 
for the job, and it would, be instructive to know the* range of reading requirements 
imposed at this stage of formal, introductory job training. 



Readability Metlvpd 
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Job Knowledge and MOS/ET 

The job knowledge method (Table 18, Columns 7, 10) is a standard, straightforward 
approach to determining the MOS reading requirements that can readily be adapted to 
use existing personnel data. In the basic paradigm (REALISTIC, Table 18, Column 7)7 it 
. calls for preparation of a job knowledge test for the MOS and administration of this and 
a standardized reading test to a full range sample of job incumbents. The reading grade 
level . requirements reported were obtained from empirical data and for the criterion, 
asserted by the researchers, that defined satisfactory job knowledge as the 25th percentile 
of job knowledge for the sample of job incumbents. 

The more administratively feasible variant of this method (READNEED, Table 18, 
Column 10) used the MOS/ET and the official minimum passing test score, for the 
criteric^ definition of job knowledge proficiency and the AFQT for ^an estimate of 
reading grade level. Since these measures already exist in Army data banks and since the 
AFQT-RGL relationship is stable and need not be repeated, this procedure is easily 
applicable to any and all MOSs. An elementary computer program for ascertaining the 
lowest AFQT level at which MOS/ET scores meet the existing criterion point, and for a 
fixed linear transformation of that AFQT level to a RGL estimate, would produce a 
simple and inexpensive means of establishing the RGL requirement for any MOS for 
which the existing AFQT and MOS/ET data occur on a common data tape. 

The illustrative MOS reading requirements generated by this method are keyed to 
the minimum passing test score criterion set by the proponent agency for the MOS in 
conjunction with the Enlisted Evaluation Center. The necessarily judgmental establish- 
ment of these criterion cutting points is based upon a^variety of considerations involving 
personnel and manpower policies, although not all are germane to the purpose of defining 
target levels of job proficiency. For the primary purpose of establishing realistic objec- 
tives and target levels of job-knowledge proficiency for an MOS, it would seem appro- 
priate to set the criterion cutoff point for this purpose independent of other factors. 

It should be noted that any written job knowledge test requires both general reading 
ability and specific job knowledge. The man taking the test must read the multiple-choice 
questions about the content of the job material. Inability to read and understand the 
question disables him from showing whether or not he possesses that item- of job 
knowledge. The consistent substantial lrelationship between reading ability and job knowl- . 
edge measures ^ (predictive and concurrent validity r= .6 in CST and concurrent validity 
r = .5 in job incumbents) indicates the importance of general reading ability. Whether 
people who have learned to read better also tend to acquire most job knowledge in 
formal training programs and in on-the-job experience, or whether general reading ability 
merely sets limits to the amount of job knowledge that can be manifested in a written 
test is moot question in these data. 

Job Sample Method 

The job sample method (Table 18, Column 8) is an empirical procedure using as 
criterion variable the hands-on performance on > an extensive sample of individually . 
administered job -tasks. Except in the case of the Supply MOS where the job tasks are 
predominantly reading tasks, this criterion makes no direct demand upon reading but is 
, presumably affected by internalized job knowledge acquired, to an unknown extent, 
tlirough prior reading. As with the reAlistic job knowledge method, the criterion \^ 
' point was judgmentally selected to include the 75% of job incumbents scoring highest on ' 
the hands-on performance measure. , /• 

Although the job sample performance nieasure is less highly related to reading 
ability than are the more verbal measures of proficiency, this method, in conjunction 
with the criterion cutoff " assumed by the researchers, yields MOS • reading requirement 
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levels that, appear reasonable. However, the costs of constructing and administering an 
extensive job-sample test to a representative sa'mpie of job incumbents in an\MOS seem 
prohibitive for all but fundamental research purposes. ^ " =■ 

Job Reading Task Me thod ^ 

The job reading task method (Table 18, Column 9) represents the most direct 
empirical approach to determining MOS reading requirements in that it takes as its 
criterion measure the reading score on the JRTT, a sample of- actual, and commonly 
used, job reading. materials. To the extent that the job reading passages constituting the 
JRTT comprise or represent all the reading tasks of the MOS, the ability to read the 
JRTT passages is the ability to perform the job reading tasks and thus to meet the MOS 
reading requirement. 

As with^'the readability method, the problem with the JRTT method is that of 
obtaining a listing of the materials that must be read in order to achieve satisfactory job 
proficiency. In the present research, reading passages were selected for theAlRTT from 
the printed materials that the job incumbents remembered reading dtfifing the previous 
month. Thus, the JRTT comprises the material most frequently reported by incumbents 
as being used in the day-to-day job performance. 

Under this method, the reading requirement for an MOS is set as the lowest reading 
gr^e level at which criterion performance on the JRTT is reached. Making allowance for 
measurement error and the subjects' limited job experience, the criterion of job reading 
proficiency was taken as the point at which 80% of the POst-CST subjects passed at least 
70% of the JRTj: items. Illustrative MOS reading requirements associated with this 
criterion level are presented in Table 18, Column 10. 

Preparation and administration costsr^Qj:.. the JRTT are substantial in that high-usage 
job reading materials must be determined by interview and obsei-vation, and both the 
resulting JRTT and a reading measure must then be administered to a representative 
sample of job incumbents in that MOS. The relationship (r = .78) between general and 
job task reading ability suggests a considerable commonality to these two reading 
measures. For the general purposes of determining MOS reading requirements, the cost 
and effort of preparing MOS-specific reading tests offers no apparent advantage beyond 
the sometimes important factor of high face validity. 



SUMMATION 

In summary, it must be explicitly recognized that the readability, job-knowledge 
job-sample, JRTT, ind PMOS/ET measures of job proficiency are incommensurate; no 
single heuristic decision rule for establishing a meaningful comparable cut-off level of 
satisfactory job proficiency has evolved for the five different methods of measuring job 
proficiency. 

Each method provides a set of reading-requirement levels coordinate with a set of 
\ job-proficiency levels. Each method us,es a different measure of job proficiency, an'd for 
four of the five such measures a different arbitrary criterion level was set to provide an 
illustratively finite reading- requirement value for that method. For eacK method, setting a 
different, judgmental, proficiency^criterion level results in a different job-literacy- 
requirement level. Accordingly, these different methods would specify the same reading 
requirement for* a job, only to ^the extent that the different criterion cutoff points 
judgmentally set on different job-proficiency dimensions, all represented equal literacy 
requirements. - 
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The REALISTIC and READNEED research presents general methods and a full 
range Oldata for determining the MOS reading requirements for any specified criterion 
level ^^ob performance in that MOS. Specific examples of the outcomes .of the 
application of these procedures have been presented for several MOSs in Table 1^, 
Columns 6 to 10. Each of the spt^'^ific reading requirement values presented in this tabic 
depends on the job-performance-criterion level adopted through the arbitrary judgment of 
the researchers for illustrative purposes. Therefore, these values should be expected to be 
comparable only to the exteixt that informed policy judgment -agrees with the present 
choice of criterion levels of job performance as representing, the Army's definition of 
satisfactory job performance. - " ' - . 
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Appendix A 



EXPERIMENTAL PASSAGES 



Passage 11. 

Going into the prone. position: 

Hold your weapon unBer one arm and crouch slowly. Feel for the ground with your 
free hand and make sure it is clear.- Lower your 'knees, one at a time, unHl your weight is 
on your knees and free hand. S^lLft your weight to your free hand anc; opposite knee. 
Raise your free leg up and back and lower it gently to the ground, feeling with your toe 
for a clear spot. Roll gently to that side and move yoiu: other leg into position the same 
way. Roll quietly into the prone position. 

Crawling: ^ 

The low crawl and high crawl are not suitable when you are very near the enemy. 
They make a shuffling noise which is too easily heard. Crawl on your hands and knees. 
Lay your weapon on the ground :by your side. With your n^^ht hand, feel for or make a 
clear spot for your knee! ^ - , 



Passage 12 

Use of the Military Police Club: 

The military police club is used only in circumstances in which it is fully justified. 
Learn through ^practice to use the club in your* left hand. This leaves your right hand free 
to handle your firearm in case of necessity. Never take the club from the belt for use 
except in an emergency. 

Holding the club: >_ , • 

To hold the club, place the thong over the left thumb, across the back of the hand, 
and grasp the grip. If the thong proves to be too large, turn the club in your hand after 
the grip is completed until the thong is tight across the back of your hand. If the club is . 
held in this fashion, your hand does not become bound to the club if an'attbrnpt is made 
to pull you off balance ^y puUthg at the club. The club normally is used as an extension 
of the arm. ! / ' 

' / ' ' ' ' 

Passage 13 

Application of Pressure Dressing: * 

The application of a sterile dressing^ with pressure to a bleeding wound helps clot, 
formation, compresses the open blood vessels, and protects the wound from further'^ 
invasion of germs. The foUov/ing procedure shojald be used when a person is wourided. 



./ 

/ 

/ 

./ 



■ / 

// 

Look for more than one wound. For example^ a missile may have come out at 
another point. The wound nvhere a missile comes out is usually larger than the one where 
it enters. • ' // - ' 

Cut the clothing and lift it away from the wound /ib avoid further contamination. 
Tearing the clothing might result in rough handling of the injured part. Do^not touch the 
wound; keep it as clean as possi.ble. If it is aheady dirty, leave it that way. Do not try to 
clean it in any way. / 

Cover the wound with a first aid'dressing anfl apply pressure to the wound, by use 
of the bandages attached to the dressing. / 



Passage 14 ^ 




Range mark gate multivibrator Ql and Q2 produces a 500-microse^condnTegatiye gate 
which is delivered to blocking oscillator Q3 and inverter Q4. The gate delivered to 
blocking oscillator Q3 is differentiated prior to arrival. The pulse corresponding to the 
leading edge is used a trigger blocking oscillator Q3, which, in turn, produces the 
touchdown marker. The touchdown marker is later, inserted at the start of the range 
mark train in place of the first range mark (which^was deleted by the range mark cancel 
pulse). Inverter Q4, which receives the complete range mark gate, has two outputs. One 
output, the inverted range mark gate, is applied to switch Q5. Upon reception of the 
range mark gate, switch Q5 gates-on free-running range mark multivibrator Q8 and-Q97' 
Simultaneously, a negative spike corresponding to the leading edge of the range mark gate 
is. developed at inverter 04, . and apphed through the RM POS variable resistor -and 
emitter follower Q6 to the range mark multivibrator. 

,/■ ■ ■ ■ ■ . \ ■ ' 

passage 15 . ^ 

Adequate protection from the elements and environmental conditions must be 
provided by means of proper storage facilities, presei*vation, packaging, packing, or a 
combination of any or all of these measures. To adequately protect most items from the 
damaging effect^s of water or water-vapors, adegliate preservation must be provided. This 
is Qlften true even though ^e item is to be stored in a warehouse provided with 
/fYT^hanical means of controlling the l^mperature and humidity. Several methods^ by 
Hvhich humidity is controlled are in use by the- military services. Use is also -wlaae of 
mechanically ventilating and dehumidifying selected sections of existing warehouses. 
^Appropriate consideration will be given to the preparation and care of items-stored under 
specific types of storage such as controlled humidity, refrigerated, and heated. The 
amount and levels of preservation, packaging, and packing will be governed by the 
specific method of storage plus the anticipated length of storage- 
Passage 16 

Radio interference suppression is the elimination or minimizing of the electrical 
disturbances which interfere with radio reception, or disclose the location of the vehicle 
to sensitive electrical detectors. It is important that vehicles with, or without radios be 
properly suppressed to prevent interference with radio reception of neighboring vehicles. 

The ignition and generating ' systems have been designed to suppress radio 
interference. Ignition system interference suppression is effected by a primairy capacitor 



in the distributor wiring harness receptacle, a resistor "suppression in the spark piugs, and 
by shielded spark plug cables. Radio interference suppression, in the generating system is 
effected by a choke, capacitor, filter and rectifier in the generator regulator assembly. 
The primary capacitor^ is housed in the distributor wiring harness receptacle and is an 
integral pax^t of the primary circuitr-Spark plugs are shielded inclividually by metallic braid 
beneath the rubber insulation. The generator regulator assembly^^ houses the capacitor, 
choke, filter and rectifier. . . , i * 



Passage 21 

,If you do not have a compass, you can find direction by other methods. 
The North Star. North of the equator, the North Star shows you true north To find 
the North Star- 
Look for the Big Dipper. The two stars at the end of the, bowl are called the 
'^pointers.'' In a straight line out from the pointers is the North Star (at about five times 
the distance between the pointers). The Big Dipper rotates slowly around the North Star, 
and does not always appear in the sarne,position. - - 

You can also use the constellation Cassiopeia. This group of five bright stars is 
shaped like a lopsided M (or W, when it is low in the sky). The North Star is straight out 
from the center star about the same distance as from the Big Dipper. Cassiopeia also 
rotates slowly around the North Stai- and -is always almost directly opposite "tl^' Big 
Dipper. ' 



Passage 22 - 

o " 

Service Headlights Adjustment. 

Position vehicle on a level floor \yith no load in vehicle. Headlights should.be 25 
feet from a smooth vertical surface. Inflate tires. to. proper pressure. 

Measure centerline of headlights from floor; draw a horizontal line at that heigbt on 
the flat surface. Draw a second line parallel to and 3" below first line. 

Draw a vertical line intersecting the first two lines at the projected centerline of the 
vehicle. . . « ' ^ 

' Measure distance betwpn two headlight centers, then divide' distance equally, on 
both' sides of centerline. D^aw a vertical line at these points, intersecting the first two 
lines. ( 

Turn headlights on ancl select high beam. Cover one light while adjusting the other. 
- Turn-.. adjusting screws in or out until beam is adjusted to a pattern as near as 
possible to that shown in figure 2-213. Adjust other headlight in same manner. 

Replacement of Sealed Beam Lamp. 

Remove three screws and lockwashers. ^ 

Passage 23 ' 

The purpose of padding a cast is to provide more comfort for the patient, to lessen 
the possibility of pressure sores, and to make it easier and safer to remove the cast. ^ 

Stockinet may be used next to. the skin as a padding material for a close-fitting and 
well-contoured cast. It should not be used alone for ^acute fractures, where there is 
excessive swelling, or immediately after an operation, since it tends to constrict and may 
impair circulation. If stockinet is used vvithout additional padding, the fact should be 
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noted with indeljble pencil on the cast, so that .whl^ii the cast is ranovod the electric 
cutter will be used with caution. ^ 4; > ^ ' 

Sheet 'cotton or webril bandage can be wrapped ovc^r the stockinet ^n. one to thivo 
layers. It . should be rolled on smoothly with the turns overlapping about one-half the 
width of the bandage. _ • ^ . 

Bony prominences are then padded with pieces of felt. 

Passage 24 • , 

In order to draw valid inferehces concernijig the overall quality of stored material, 
the sample units sjt^lccted for examination and testing should be representatiye^r the 
*^population*' of supplies from which this sanlple has been drawn. :The **popiption'' may 
be a lot established at time of manufacture, a group of " these lots similar in charac- 
teristics, or all the mateHal stored under the same conditions: Selection ^of siimples shall 
be accomplished in a manner that will assure that each unit in the \oi has an, equal 
chance of being included. Xbat is, items must be selected without regard to their quality. 
Biased sampling methods must be avoided, such as selecting items from one location, 
selecting items that appear to be either good or defective, etc. Obvious defectives should 
-^be identified but not removed until the sample has been .selected. Tables of random 
numbers, or similar devices, should ^be used for the selection ^process. % 



Passage 25 , ^ ■ 

Rear area protection (RAP) encompasses. two broad functions—rear area seburity and 
area damage control. Military police are priniarily concisrned with rear area* security. The 
logical and normal deployment of military police elements. in a combat area makes them 
ideal for RAP operations. Their "^areas of interest normally coincide with j:he enemy's area 
of interest. In addition, military police normally will be the firs? element :;o investigate an 
incident or to.be on the scene. Their inhereht mobUity coupled with excellent communi- 
cations provides a capability to construct rapidly an integrated JR. AP element on the spot.. 

Whenever po*ssible, military police rear area security responsibilities are programmed 
to provide reconnaissance and escort platoons. This approacH provides an area com- 
mander* an essential tool for RAP operations and simultaneously^ provid(5s military poHce 
with tl^e minimum equipment to perform their reconnaissance ^nd escort responsibilities. 
A detailed discussion of rear area protection will be found in FM 19-45-1. 

Passage 26 ' . / ^ 

Flagging action wQl be removed when the individual is undergoing punishment under 
Article 15 or during a suspension -of such punishment imposed, when the commander has 
indicated that the nonjudicial punishment will be administered in a manner that will 
preclude its consideration in a possible promotion or other favorable action. Except, as 
indicated above, flagging action ^yal not be removed while an individual is Undergoing 
punishment, imposed under Article 15, UCMJ, or sentence of courts-martial, or during th^ 
term of anV. ^•suspension of punishment imposed. An individual is considered to be 
undergoing punishment with respect to any forfeiture or detention of. pay as folltiws:^^ 

When forfeiture or detention of pay/has been imposed by sentence of court;martial, 
.until the day following the last day of the period , of forfeiture or detention of pay 
specified' in the sentence of the court as finally approved, 'if all other portions of the 
sentence have been completely executed. 



Appendix B 

STRUCTURAL PROPERTY VALUES OF 
EXPERIMENTAL PASSAGES 



Structural Property 



Number of sentences 
Words per sentence 
Number of independent clauses 
Words per independent clause 
^ Number of one-syllable words 
Number of difficult words 
Number of different difficult words 10 
Number of different words 
Number of three-or-more- 
, syllable words'^' 
Total number of syllables 
Total number of letters 
Syllables per sentence 
Letters per sentence 
Number of words of seven or 

more letters , 
Number of different thr^e-or- 

more syllable words 





/ 

r 




Passage* - 


11 


12 


13 


14^' 


15 


J 6 


* 21 


22 


23 


24 


25 


26 


1 4 
1 1 


Q 


1 1 


8 


7 


8 


11 


14 


6 


6 


r 

9 


4 


11 


17 


13 


' 19' 


21 


19 


14 


11 


25 


25 


17 


,38 


18 


10 


15 


8 


7 ' 


' 8 


11 


17 


6 


6 


9 


4 


8 


15 


19 


19 


21 


T9 


14 


9 


25 


25 


17 


.38 


124„ 


122 


106 


91 


79 


68 


il4 


93 


99 


8?' 


68 


78 


15 


18 


21 


51 


/49 


70 


18 


■33^ 


37 _4§ 


74 


48 


; 10^ 




17 


34 


34 


43 


11 


19 


S-r" 35 


.46 


26 


79 


76 


91 


" 69 


89 


70 


75 


87 


94 


95 


78 


69 


^ 8 , 


.8 . 


13 


25 


40 


62 , 


9 


24 


15 


, 33 


57' 


36 


184 


191 


212 


251 


289 


323 


200 


236 


227 ■ 


260 


319" 


276 


608 


604 


664 


730 


819 „ 


894 


631 


741 


692" 


748 


862 


774 


.1.3 


21 


18 


31 


41 


40 


18 


17 .. 


38, 


43 


35 


69 


43 


57 


55 


91 


117 


112 


57 


53 


115 


125 


96 


194 


9 


15 


33 


38 


54 


s^68 


20 


40 


35 


40 


60 


52 


4 


7 


12 


17 


31 


32 


8 


16 


13 


25 


-32 


22 



Each iDassage contained 150 words. 
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Appendix C 



DATA USED TO ASSIGiN RGL SCORE TO 
EACH EXPERIMENTAL PASSAGE 



This appendix presents the data used to assign :a reading grade level score to each of 
the experimental passages. Data are presented- for the 35%, correct criterion level, used for 
the FORCAST index, and for the 30, 40, and 45% correct criterion . levels, for comparison 
purposes. For each passage, tabulated data show the percentage of subjects at each 
reading grade level scoring at or above the criterion level on the cloze test for that 
passage. The'Ns indicate the number of subjects In each reading grade lev^l who were 
" tested on the set of six passages listed to the right of the N column and on whom the 
tabulated percentages are based. * • 
■ ' . ^- ■ ■ 

' . Table C-l 



Percentage of Subjects at Each Reading Grade Level 
Meeting Indicated Cloze Criterion 



RGL 


N 


- ■ 


First Set of Passages 


N 


Second Set of Passages 


n 


12 


13 


14 


15 


16 


21 


22 


23 


24 


25^ 


26 



30% Criterion Level 



12.0-12.9. 


35 


100 


TOO 


97 


40 


89 


83 


25 


100 


.96 


96 


68 


80 


88 


11.0-11.9 


50 


96 


98 


- 98 


38 


70 


58 


49 


96 


86 


94 


63 - 


57 


51 


10.0-10.9 


17 


94 


94 


100 


29 


53 


59 


14 


100 


79 


93 


50 


57 


64 


9.0^9.9 


14 


100 


93 


100 


36 


50 


57 


27 


93 


56 


70 


44 


22 


33 


8.0-8.9 


26 


88 


81 


96 


4 


31 


19 


30 


100 


63 


77 


30 


23 


23 


7.0-7,9 


13 


77 


85 


69 


23 


31 


8 


15 


100 


60 


73 


13 


20 


27 


6.0-6.9 


14 


50 


71 


64 


14 


7 


0 


.12 


75 


25 


33 


0 


0 


25 


5.0-5.9 


14 


43 


50 


36 


^7 


0 


0 


12 


67 


8 


8 


0 


0 


8 


Below 5.0 


17 


29 


35 


35 


0 


0 


0 


11 


54 


18 


9 


0 


9 


9 



35% Criterion Level 



12.0-12.9 


35 


97 


100 


97 


3^1 


66 


63 


25 


100 


96 


96 


60 


64 


72 


11.0-11.9 


50 


86 


96 


96 


26 


52 


44 


49 


96 


71 


69 


35 


33 


24 


10.0-10.9 


17 


76 


94 


94' 


' 12 


29 


35 


14 


100 


71 


86 


29 


29 


43 


9.0-9.9 ^ 


14 


100 


93 


93 


21 


21 


29 


27 


93 


48 


59 


15 


7 


22 


8.0-8.9 ' 


26 


73 


73 


81 


4 


12. 


4 


> 30 


100 


23 ' 


37 


10 


0 


17 


7.0-7.9 


.13 


69 


62 


54 


8 


8 


8 


15 


* 87 


40 


20 


0 


.0 


7 


6.0-6.9 


14 


' 29 


36 


36 


7 


0 


0 


12 


75 


17 


8 


0 


0 


17 


5.0-5.9 


"14 


29 


43 


21 


0 


0 


0 


12 


25 


8 


0 


' 0 


0 


8 


Below 5.0 


17 


12 


24 


12 


0 


0 


0 


11 


36 


0 


0 


0 


0 
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Table C-1 (Continued) 



Percentage of Subjects at Each Reading Grade Leva! 
Meeting Indicated Cloze Criterion 









First Set of Passages 








Second Set of Passages 




RGL 


N 














N 














11 


12 


13 


14 


15 


16 


21 


22 


23 




25 


26 


40% Criterion Level * 




12.0-12.9 


35 


97 


100 


97 


31 


'54 


46 


25 


100 


92 


92 


52 


60 


68 


11.0-11.9 . 


50 


80 


96 


96 


10. 


34 


32 


49 


94 


65 


.55. . 


^6 


24 


14 


10.0-10.9 


17 


71 


94 


94 


6 


18 


24 


14 


100 


64 


79 




14 


21 


9.0-9.9 


14 • 


93 


86 


93 


21 


21 


14 


27 


93 


41 


52 


0 


- 4 


18 


8.0-8.9 


26 


54 


«69 


73 


4 


12 


0 


30 


97 


23 


23 


10 




10 


7.0-^.9 


13 


62 


62 


54 


8 


8 


0' 


15 


80 


20 


7 




\i 


7 


6.0-6.9 


14 


21 


36 


29 


0 


0 


0 


12 


58 


17 


.8 


I' 


0 


17 


5.0-5.9 


14 


14 


36 


21 


0 


0 


o' 


12 


8 


0 


0 


0 


0 ^ 


0 


Below 5.0 


17 


12 


12 


6 


0 


0 


0 


11 


27 


0 


0 


0 


0 


9 


45% Criterion Level 


12.0-12.9 


35 


94 


97 


91 


14 


31 


29 


25 


96 


72 


72 


24 . 


36 


44 


11.0-11.9 


50 


60 


. 88 


88 


2 


18 


20 


49 


92 


37 


41 


8 


12 


10 


10.0-10.9 


17, 


53 


71 


65 


0 


0 


6 


14 


93 


36 


36 


0 


0 


7 


9.0-9.9 


14 


50 


71 


50 


14 


14 


0 . 


27 


90^ 


22 


15 


0 


0 


11 


8.0-8.9 


26 


31 


42 


46. 


4 


4 


0 


30 


73 


7 


13 


3 


0 


7 - 


7.0-7.9 , 


13 


38 


31 


38 


0 


0 


0 


15 


60 


13 


7 


0 


0 


1 


6.0-6.9 


14 


14 


14 


7 


0 


0 


0 


12 


33 


8 


8 


0 


0 


8 


5.0-5.9 


14 


7 


0 


7 


0 


0 


0 


12 


8 


0 


0 


0 


0 


0 


Below 5.0 


17 


6 


6 


0 


0 


0 


0 


11 


18 


0 


0 


0 


0 


9 
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Appendix D 



MEAN PERCENT CORRECT CLOZE SCORE FOR 
CLOZE TEST VARIATIONS 









Variation 


















Cloze Passage 














1 


2 




4 


5 


. 11 


35 




. 41 


' 38 


53 


12 


45 


47 


43 


50 


49 


13 


.50 


51 


48 


39 


40 


14 


24 


17 


15 


24 


17 


15 


31 


27 


32 


23 


23 


16 


18 


27 


27 


22 


32 


21 


54 


52 


62 


51 


51 


\22 


■35 


40 


35 


25 


32 ' 


23 


31 


40 


■ 36- 


34 


34\ 


24 


25 


20 


33 


20 


21 


25 


27 


18 


28 


24 


18 


26 


17 


24 


25 


40 


18 



Appendix E 

MULTIPLE CORRELATIONS OF SELECTED SETS OF 
PREDICTORS WITH-MEAN CLOZE SCORE 



Predictor Sets'' 



Words per Sentence (2) + One-Syllable Words (5) . ,87 

Words per Sentence (2) + Difficult Woi-ds (6) -.89 

Wojds per Sentence (2) + Different Difficult Words (7) • -.89 

Words per Sentence (2) + Polysyllabic Words (9) -.82 

Words per Sentence (2) + Total Syllables (10) -.85 

Words per Sentence (2) + Sey.en-or-more Letter Words ( 14) -.81 

Words per Sentence (2) + Different Polysyllabic Words ( 1 5) -.84 

Words per Sentence (2) + Total Syllables (10)..+ Total Letters (11) -.85 
Words per Independent Clause (4) + One-Syllable Words (5) .'■ .87 

Words per In depen dent Clause (4) + Total Syllables (10) -.85 



^Predictor variables are numbered in accordance with Table 4. 



^;• Appendix F 

MANUALS AND REGULATIONS SAMPLED TO 
DETERMINE READING DEMANDS OF SEVEN MOSs' 



Manuals 


N 




Manuals 


N 


1 1B20 Light Weapons Infantryman 




63B20 Wheel Vehicle ^^Qzhm\Q (Cont) 




TM 


9-1340-214-12 ' \ 


7 


TM 


9-2320-244-20 


3 


TM 


11-5805-20M2 


3 


TM 


10-3930.242-12 


6- 


TM 


11-5855-203-13 , 


8 


TM 


21-305 


4 


FM 


5-1^ 


6 


TM 


38-750 


9 


FM 


20-32 ^ ^ . 


7 


AR 


310-1 


3 


FM 


20-33 


^ 4. 


AR 


. 750-5 


3 


FM 


21-26 


12 


71H20 Personnel Specialist 




FM 


21-75 


y 12 


. AR 


55-46 


3 


FM 


23-8 . 


3 


AR 


210-10 


o 

d 


FM 


23-9 


4 


AR 


310-1 


3 


FM 


* 23-12 


. 5. 


AR 


310-10 


A 


FM 


23-23 ^ 


3. 


. AR 


340-15 


7 


FM 


23-30 


«j 


AR 


380-5 


3 


FM 


23-31 


a 
o 


AR 


600-10 , 


3 


FM 


23-65 


11 


AR 


600-31 


6 


FM 


23-67 


3 


AR 


600-200 


7 


FM 


23-71 


3 


AF^ 


601-280 


3 


FM 


24-18 


4 ^ 












AR 


606-5 


3 


26D20 Ground Control Radar Repairman 




AR 


608-3 


3 


TM 


1V5840-281-ESC 


3 


AR 


611-101 


3 


. TM 


11-5840-281-15 


38 


AR 


611-103 


4 


' ' TM 


11-5840-293-12 


13. 


AR 


61M12 


3 


TM- 


11-5895-468-12 , 


7 


AR- 


611-201^ 


3 


TM 


11-5895-474-12 


6 


AR 


614-200' 


3 


TM 


- 38-750 


11 


AR 


630-5 ^ . ' 


3 








AR . 


635-200 


3 


63B20 Wheel Vehicle Mechanic 




AR 


640-10 


3 


TM 


5-4310-26-15 


3 


AR 


672-5-1 


3 


TM 


.9-243 


9 


AR 


680-1 


6 


TM 


9-2320-209-20 


26 


680-20 






AR 


3, 


TM 


9-2320-21 MO 


4 




TM 


'9-2320-211-20 


^ 24 


DA Pam 600^8 


3 


TM 


9-2320-218-20 


22 


DA Pam 608-2 


3 








DAPam61M 


4 
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Manuals 



Manuals 



76Y20 Armorer/Unit Supply Specialist 
s TM 55-431 0-200-25P ^ 
, TM 9-1005-223-20 
TM 9-2820 
TM ' ^38-750 
. TM ' 743-200 ' ' ' 

FM 20-1 5 
FM 21-11 
FM, 21-15. 

AR ^ 210-130 

.AR ^ 220-10 

AR 3101 

AR 340-15 

AR 340-18-1 ' 

AR 385-55 

AR 700-84 
•* « 

AR 700-87 

AR 735-1 1 

AR 735-35 

AR 746-10 

AR 750-1 

DA Pam 310^4 ' 
DA Ram ai 0-6 , 

91 B20 Medical Specialist 




95B20 Military Policeman 
FM 19-4 



3 
5 
8 
3 
3 

3 
'3 



FM 
FM 
FM 
FM 

FM 
FM 
FM 
FM 
FM 

FM 

.FM 
FM 
FM, 

fm' 

FM 
FM 
FM 
FM 
FM 
FM 

AR 
AR 



19-5 
19-10 
19-15 
19f25 

19-30 
19-40 

1 9- 50 

20- 32 

21- 11 

21-26 

21- 75 

22- 5 

23- 9 
23-31 

23- 35 

24- 1 
24-18 
27-10 
3M6 
31-23 

190-22 
190-45 



DA Pam 360-530 
MCM 



TM 


8-230 


30 


TB PMG 


TM 


8-231 


3 




TM 


8-273 


12. 




FM 


8-35 


. - ' 5 




FM 


21-10 


6' 




FM 


2M1 


12 




AR 


40-4^19 


3 




AR 


'40-425 


3 




AR 


40-562 


3 




AR 


600-6 


3 




AR 


735-35 


3 




TB MED 246 


3 





3 
17 
7 
7 

10 

6 
5 
5 
3 
7 

10 
3 
4 

5 
3 

4 

3' 
,-4 
6 
3 
4 

3 

3 ■ 
7 
3 
3 



^See Table 7. 
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2 DIM OASO MANPOWCK IPPtGRI 

I CHF DCF NUC AGCY ATTN QGC L I B BR 

I DIP kSEG WASH., O.C. 20305 

I DIR DASD HANPDk'ER C RESERVE AFFAIRS 

1 UFC OF THE ASST SEC OF DEF {HtEJ (QAAI ATTN M BIEGEL 

2 CDMUR FLD CDHD DEF NUC AGCY S AND I A BASE ATTN FCTG7 
2 NASA SCI t TECH INFD FACILITY CDLLEGE PARK HD 

I CINC US EUROPEAN CDMD MTN SUPPDRT PLANS BR J3 

I CG US ARHY JAPAN APG %3A3 SAN FRAN ATTN G3 
10 CG USA FORCES SDUThERN CGmD ATTN SCARCD CZ 
. 2 CG US ARMY EURDPE APrf D9AD3 NY ATTN OPNS DIV 

I CD ARHY TRANS RES COko FT EUSTiS ATTN TECH Lift (; 

I CG USA AD CDHD ENT AF B ATTN AOGPA COLD 

6 CG 1ST ARKY ATTN DCSOT FT KEADE MD 

I CG 3RD ARMY ATTN DCSDT FT MCPHERSDN 

4 CG SIXTH ARHY PRES DF SAN FRAN ATTN AH0PS-T2 

I CG EUSA ATTN AG-Ac APO 963DI SAN FRAN 

1 DIR HEL APG HD 

2 ENGNS PSYCHOL LAB PlUNEERiNt RES DIV ARHY NATICK LABS NATICK HASS 

4 TECH LIB ARHY NATICK LABS NAT ICK HASS 

^ INST OF LAND CBT ATTN TECH LIB FT BELVDIR VA 

I CO FT HUACHUCA SPT COHO USA ATTN TECH flEF LIB 

1 SIXTH USA LIB DEPOT Rl DG H 13 U PRES OF SAN FRAN 

5 CG FT ORO ATTN G3 rNG_DlV 

I CO HO-ARHY ENLlSTEf3<EVAL CTR ^T BENJ HARRISON 

1 LIB DEF SUPPLY AGCY CAHERON STA VA 

2 CO USA CDC AG AGCY FT BENJ HARRISON I NO 
I Cf. ARMY COC IN.- AGY FT BENNING 

I CO ARHY CDC ARHOR AGY FT KNOX \ 
I OIR OF IVTERN TNG USA LOG HGT CTR FT LEE 

3 CO USA CTR (FA) ATTN AKPSITC-TT FT SILL 
CG USA TnG CTR C FT LEONAROlwDOD ATTN ACOFS C3 
CG USA INF CTR ATTN AJiqT-T FT BENNING 
CG USA TNG CTR INF ATTN ACOFS G3 FT OlX 
CG USA TNG CTR ATTN ACOFS GJ FT JACKSON 
CG USA TNG CTR INF ATTN ACOFS G3 FT LEWIS 
CG USA TNG CTR INF t FT ORO ATTN ACOFS G3 
CG USA TNn CTR INF ATTN ACOpS G3 FT PaK 
CO USA HED TNG CTR ATTN OlR OF TNG FT SAH HOUSTON 
CG oSA AD CTR ATTN G3 FT BLiSS. . 
CG USA TNG CTR INF ATTN ACOFS G3 FT CAHPBELL 
LIB ARHY MAR COLL CARLISLE BKS 

COHC*' CDHD ♦ GEN ST*PP CO HT LEAVENWORTH ATTN ARCHIVES 
OIR OF HILIT PSYCHOL LORSHP US HILIT ACAD WEST POINT 
US MILIT ACAD WEST POINT ATTN LIB 
COHOT ARHY *VN SCH ATTN OIR OF INSTR FT RUCKER 
COHOT ARHY SECUR AGY TNG CTR ♦ SCH FT OEVENS ATTN LlB 
STIhSON lib hED FLD SERV SCH BROOKE ARHY HED CTR FT SAH HOUSTON 
COHU? THE ARHOR SCH ATTN DDI FT KNOX 

COMOT USA CHAPLAIN SCH ATTN DDI FT HAMILTON \ 
COMOT USA FIN SCH ATTN ChF OOC DEV LIT PL N DIV 0001 InO 
USA FINANCE SCH Ft BENJ HARRISON ATTN EOUC ADV 
COHOT AOJ GEN SCH FT BENJ HARRISON ATTN EOUC \Qy 
COMOT. USAl S ATTN EOUC ADV FT BENNING 
COHOT USAIS ATTN A J I I S-O- E PRO FT BENNING 
HQ US ARMY AOJ GEN SCH FT BENJ HARRISON ATT COHOT 
LIB ARHY OH SCH FT LEE 

COHC^ USA OM SCH FT '^.EE ATTN ErpC ADV 
COHOr ARHY TRANS SCH FT EUSTii EOUC ADV 

CO USA SEC AGY TNG CTR C SCH ATTN :\TEV RSCH ADV FT OEVENS 
COMOT USA HIL POL^^.E SCH ATTN PLNS CPROG OOOI FT GORDON 
COMOT US ARHY S0o;hEASTERN SIG SCH ATTN EOUC AOV GOROC- 
COHOT USA AO ^CH ATTN 001 FT BLISS 

CO USA QRO CTR f. SCH OFC OF OPS ATTN AHBN-0 AUG HO 
*.5ST COHOT ARHY Al R OEF SCH FT BLISS ATTN CLASSF TECH LIB 
COHOT OEF INTELL SCH ATTN SiCAS OEpT 
COMOT ARMED FORCES STAFF CGLL NORFOLK 
COMOT USA SIG CTR C SCM ATTN 001 FT MONMOUTH 
COHOT JUD-.E ADVOCATE GENERALS SCh U OF VA 
DPTV COMOT USA AVN SCH ELEMENT. GA 

USA AVN SC>( ELEMENT QFC OF Ol/oF INSTR ATTN EOUL ADV GA 
EOUC CONSLf ARMY HIlIT PQLICJE SCH FT GORDON 
COHOT US^ ENGR SCH ATTN ^UC AOV AHBBES-gA FT BEL VOIR 
COMOT USA SCH EUROPE ATTN EOUC ADV APO 09172 NY 
OFC OF DOCTRINE OFV LIT £ PLNS USA ARMOR SCH ATTN AMBAAS-OM 
COHOT ARHY AVN SCH FT RUCkER ATTN EOUC AOV 
DIR OF INSTR US MIL ACAO WEST POINT N/ 
OIR Of HlLlT INSTR US HiLiT ACAO WESJ POINT 
CnnOT OEF HGT SCH'FJ BELVOIR 

COMOT USA HSL C HUN CTR i SCH ATTN CHF OFC OF OPS REOSTQNE ARSNL 
COMOT US WAC SCH US WAC CTR ATTN AJMCT(FT MCCLELLAN- 
HO ABEPOEEN PG ATTN TECH LIB 

CO USA INTELL. CTR C SCH ATTN OIR OF ACADEMIC OPS FT HUACHU'^A 
CO USA INTELL CTR t SCH ATTN D|R OF OOC C LIT FT HuACHUCA 
COMOT USA CtGSC OFC OF ChF OF RESIDENT INSTR FT LEAVENWORTH 
COHOT USA CA SCH ATTN OFC OF OOCTRlNE OFVEL LIT C PLNS FT »W AGO 
COMOT USA CA SCH ATTN 001 FT BRAGG 
COMOT USA CA SCH ATTN LlB FT BRAGG 
COMDT USA iNST FOR HlL ASSIST ATTN 001 FT BRAGG 
COMOr USA FLD ARTY SCH ATTN DDI FT SILL 
COMOT USA TRANS SCH ATTN OlR tlF POC C LIT FT EUSTIS 
COMOT USA TKANS SCH ATTN LIB FT EUSTIS 
COHOT USA C6GSC AT T N 'AT SC S-D J (SPWARI 

COMOT ARMY QM SCH OFC OIR OF NONRESID ACTV/ ATTN TNG MEDIA OtV VA 
CCMOT USA ARTY SCH ATTN LIB FT SILL 
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I CG USA SCH C TNG CTR /TTN ACOFS G3 FT GORDON 
jl DIR OF GRAD STUD 6 RSCH ATTN BEHAV SCl REP USACtGSC 
I COHOT USA AO SCH ATTN AKBAAS-DL-EA FT BLISS 
I CG USA SIG CTR t SCH ATTN ATSSC-OP-COD FT MONMOUTH 
I SECY OF ARHY, PENTAGON 
I OCS-PERS OA ATTN CHF/ C*S DIV 

1 OIR OF PERS STUDIES /c RSCH OOCSr^::R DA WASH DC 

2 ACSFOR DA ATTN CHF JNG DIV WASf^ DC 

I HO ARHY HAT CDHD R*0 ORCTE ATTN AMCRO-RC 

I US ARMY BEHAVIOR L SVS RSCH LAB ATTNCRO-AR ARL VA 

I OPO PERS HGT DEV OFC ATTN HOS SEC (NEW EQUIP) DPDHD 

1 PROVOST MARSHAL GEl^ OA 

2 OIR CIVIL AFFAIRS ORCtE OOCSOPS 
I OFC RESERVE CDHPON OA 

12 AOHIN DOC ATTN: TCA (HEALYl UHERON STA ALEX., VA , 2231* 
1 CHF OF R + 0 OA ATTN CHF TECH ♦ INOSTR LIAISON OFC 
1 CO USA. COC HEO SERV AGCY FT SAH HOUSTON 

1 USA BEHAVIOR & SYS RSCH LAB ATTN CRO-AIC ARL VA 

2 TNG t OEVEL OiV OOCSPERS 

1 CAREER MGT BR ATTN R OETtENHE CAMERON STA ALEX VA 

1 USA LIB OIV-TAGO ATTN ASOIRS 

15 CG CONARC ATTN ATIT-STH FT HONROE \ 

2 CG CONARC ATTN LIB FT HONROE 

1 CO ARHY CBT OEVEL COHO hILIT POLICE AGY FT GOROGN 

1 CHF USA AC; HRU FT BLISS 

1 CHF USA ARMOR HRU FT KNOX 

1 CHF USA AVN HRU FT RUCKER 

1 CHF USA INF HftU FT BENNING 

1 CHF USA TNG CTR HRU PRES OF MONTEREY 

1 1ST ARMORED OIV HQ t HQ CO FT HOOD ATTN AC OF S G? 

3 CG ARHY ARHOR CTR FT KNOX ATTN G3 AIBKGV 

3 CG 82N0 A8N iNF OIV ATTN ACGFS G3 FT BRAGG 
1 CO I97TH INF BRGO FT BENNING ATTN S3 
5 CG 1ST INF OIV ATTN ACOfS G3 FT RILEY 

1 CO USA PARTIC GP USA TNG DEVICE CTR FLA 

2 OA OFC OF ASST CHF OF STAFF FOR COMH-ELCT ATTN CETS-6 WASH 
1 OIR ARMY LIB PENTAGON 

1 CHF OF HILIT HiST OA ATTN GEN REF BR 

1 CO USA lOTH SPEC FORCES GP FT OEVENS 

1 US ARHY GEN EOUlP ATTN TECH LIB FT. LEE 
10 CG in CORPS t FT HOOO ATTN G3 SEC FT HOOO 
30 CO IST ARhOREO OIV ATTN G3 SEC FT HOOO 1 
30 CG 20 ARH;]RE0 OIV ATTN G3 SEC FT HOOD 

1 CG USAFAC t FT SILL AV^N AKPSIGT-TNTN 
20 CO III CORPS ARTY ATTN G3 SEC FT SILL 

I RSCH CONTRACTS L GRANTS BR ARO \ 
o\ fiESD|-ARD OFC ChF OF RtO WASH OC 

1 CHF pF RCO OA ATTN SCI INFO BR RSCH SPT OIV WASH DC 

1 CG USAFACFS ATTN AKPSIAC-AS FT SILL 

1 CINC US ATLANTIC FLT CODE 312A USN BASE NORFOLK 

I CDP TNG COMMAND US PACIFIC FLT SAN 0 1 C^O 

5 TECH LIB PERS llB 1 .7 OF NAV PERS ARL ANNEX 

3 OIR PERS RES OIV BUr .OF NAV PERS \ 

1 TECH LIB BUR OF SHiPS CODE 2 1 OL NAVY DEpT 

3 CO f OIR NAV TNG DEVICE CTR ORLANDO ATTN ffCH LIB 

2 CO FLT TNG CTR NAV BASE NEWPORT . \ 
1 CO FLEET TNG CTR US NAV STA SAN DIEGO 

1. PRES NAV WAR COLL NEWPORT ATTN MAHAN LIB ^ 

3 CO S6RV SCH CUMO NAV TNG CTR SAN OIEGO 

1 CHF OF NAVL RSCH PERS L TNG BR (CODE ASBI ARL VA 

1 OIR US NAV RFS LAB ATTN CODE 5120 

1 OIR NAVAL RSCH LAB «TTN Lid CODE 2029 WASh OC 

i CHF OF NAV AIR TNG TNC RES DEPT NAV AlR STA PENSACOLa 

3 OIC NAV PERS RES ACTVY SAN OIEGO 

1 OIR PERS RES LA& NAV PtRS PROGRAM SUPPORT ACTIVITY HASH NAV YD 
S COMOT MARINE CORPS HQ HARlNE CORPS ATTN CODE AO- 1 fl 
1 OIR MARINE CORPS EOUC CTR HARINE CORPS SCl^ OUANTICO 

1 US MARINE CORPS HOS HIST RE? LIB ATTN HRS JADOT 

2 'COHOT HQS JTH NAV DIST ATTN EOUC ADV NEW ORLEANS 
, 1 ChF of NAV AIR TECH TNG NAV AIR STA HEHPHfS 

1 CHP DFCR PtRS RES ♦ REVIEW BR COAST GUARO HQ 

1 CD US CCAST GUARD TNG CTR GOVERNORS ISLAND NY 

1 CD IIS COAST GUARD TNG GTR C/^PE HAY NJ 

1 CD US COAST '•-'JARp TNG CTR C SUP CTR ALAMEDA CALIF 

1 CO US COAST . ^ARD iNST DKLA CITY DKLA 

1 CO US COAST. GUARD REb TNG CTR YDRKTOWN VA 

1 SUPT US COAST GUARD ACAD NEW LONDON CONN 

1 AIR TNG COMD/XPT RANDOLPH AFB . 

I TECH OIR TECH TNG DIV(HRD) AFH^L LOWRY AFB COLO 

1 CHF SCI DIV ORCTE SCI ♦• TECH OCS R*D HQ AIR FORCf AFRSTA 

1 CHF DF PERS RES BR ORCTE OF CIVILIAN PERS OCS-PERS HQ AlR FORCE 

\ CHF ANAL OIV (AFPOPL (R) OlR DF PERSONNEL PLANNING HQS USAF 

1 ATC ATXRQ RANDOLPH AFB 

2 HILIT TNG CTR OPE LACKLAND AFB 
1 AHO AHRH BRDDKS hk i\ TEXAS 

1 HQS ATC DCS/IECH TNG {ATTHS) RANDOLPH AFB 
1 USA^A DIR DF ThE LIB USAF ACAO CDLD 
1 TECH TNG CTR ILM TC/ DP - l-L U LOWftY AFB 

3 CIA ATTN CRS/ADO STANDARD DIST 

1 SYS EVAL DIV RES DIRECTORATE OOD-OCO t^ENTAGON 

I DEPT DF STATE BUR OF INISL * RES EXTERr^AL RES STaFT 

3 U!» INFO AGYl lai L PROCURLHEMT LIB 

1 SCI INFO EXCH WASHINGTON 

2 CHF. MGt t G^N TNG OJv T« 200 FAA WASH OC 
1 EOUC MEDIA BR OE HEW ATTN T 0 CLEMENS 



1 OFC OF INTERNATL >MO PLANNING L EV&L BR MO W&Sk OC 

2 ERJC DE XASH DC 

1 CONSOL FED LAW ENFORCEMENT TnG CTR WASH OC 

2 SYS DEVCL CORP SANTA MONlCi^ ATTN Llfl 
2 OUNLAP ♦ ASSOC InC OARIEN ATTN LIB 

I RAC ATTN LIP MCLEAN VA 

1 RAND CORP MAShINGTON ATTN LIB 

2 ELhCT PERS RSCH GP U OF SOUTHERN CAMF 

I COLUMRIA U ELEC RES LaBS AVTN TECM EDITOR 

1 HITRE CORP BEDFORD MASS ATTN LIB 

2 LEARNING RCD CTR U OF PiTTS ATTN DIR 

1 HUMAN SCI RES INC MCLEAN V/ 

2 'TECh info CTR ENGNR DATA SEkV N AMER AVN IHC COLUMBUS 0 
I RAYTHEON SERV CO ATTN L IBN BUrI. I NG TON MASS 

I GEN DYNAMICS POMONA DIV ATTN LIB DIV CALIF 

I OIR CTR FOR RES ON LEARNING ♦ TEaChING U OF MICH 

J CANADIAN JOINT STAFF OFC OF OEF RES HEMRER t^ASHlNGTDN 

3 CANADIAN ARMY STAFF .WASHINGTON ATTN GSD2 TNG 
I AMER INSTSFOR RSCh ATTN LiBN PA 

3 MATRIX RSCh CQ FALLS ChURCh VA 

I GE CO WASH 0 C 

I ROMLANO ♦ CO HaODONFIELD NJaTTN PRES 

. I AMEft PSYChOL ASSnC WASHINGTON ATTN PSYCHOL ABS7R 

1 EnGNR lib FAIrChILO hILLER republic AVN DIV FARMINGDALE N 

2 ASSOC OIR CDC TNG PROC ATLANTA GA 

2 HASH MILITARY SYS TECh LIB DIV BEThESOa MD 

2 Chrysler corp oef enGr attn dr h berman Detroit 

I OR H. SHOEMAKER OlR TNG RSC'h'GP NY 

I U OF MiNN OEPT OP INDUST EOUC ATTN R E KUhL 

I CHF processing DIV DUKE U LIB 

I I OF Calif gen lib docu oept 

I (LOR I OA STATE U LIB GIFTS * E XCH 

1 PSYCHOL LIB HARVARD UnIV CAMBRIDGE 

1 U OF^ ILL LIB SER DEPT. 

2 U OF KANSAS LIB PERIDOUaL DEPT 
I U OF NEBRASKA LIHS ACQ DEPT 

1 OHIO STATE U LIBS GIFT ♦ EXCh 01 V 

1 PENNA STATE \J PATTEE LIB DOCU DESK 

1 PURDUE U UBS PERIODICALS ChECMNG FILES 

1 STANFORD U LIBS DOCU LIB . 

1 L IBN.U ^F TEXAS 



SYRACUSE U LIB SER UIV 

SERlAlS REC UnIVOF HINN MINNEAPOLIS 

STATE U OF lOwA LIBS SER ACQ 

NO CAROLINA STATt COLL ON HiLL LIB 

BOSTON U LIBS ACQ DtV 

g OF MICH I I as SER DIV 

fiP.DWN U LI B 

COlUHeiA U LIBS ClOCU ACQ 
DiR JOINT U LIBS NASHVILLE 
U OF DENVER MARY REED LIB 

LIB GEO MASH UNIV ATTN SPEC CULL DEPT HASH OC 
LIB OF CONGRESS CHF OF exCh ♦ GIFT OIV 
U OF PGh DOCU LIBN = ^ 

CATHOLIC U LIB EDUC C PSyChOL LIB WASH DC 
U OF KY MARGARET I K InG LIB 
U OF LOUISVILLE LIB BELKNAP CAMPUS 
GEORGETOWN U LIB SER OEPT WASH ot 
LIBS COLO STATE U A17N DOC LIBN FT COLLINS 
DR G BEVE» COLUMBIA U 
DR J BORMUTH UNl<y_OF CHICAGO 
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OR E 
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